2024-09-10T18:36:02.822 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-10T18:36:02.836 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T18:36:02.919 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257 branch: wip-guits-quincy-2024-09-09-1810-quincy description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} email: gabrioux@ibm.com first_in_suite: false job_id: '7900257' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-guits-quincy-2024-09-09-1810-quincy ceph: conf: client: debug ms: 1 global: mon election default strategy: 3 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon warn on pool no app: false osd: debug ms: 1 debug osd: 20 osd class default list: '*' osd class load list: '*' osd shutdown pgref assert: true flavor: default log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - reached quota - but it is still running - overall HEALTH_ - \(POOL_FULL\) - \(SMALLER_PGP_NUM\) - \(CACHE_POOL_NO_HIT_SET\) - \(CACHE_POOL_NEAR_FULL\) - \(POOL_APP_NOT_ENABLED\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} cephadm: cephadm_mode: root install: ceph: flavor: default sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 workunit: branch: quincy-orch-centos-9-teuth sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d owner: scheduled_gabrioux@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - ceph.rgw.foo.a - node-exporter.a - alertmanager.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - grafana.a - node-exporter.b - ceph.iscsi.iscsi.a seed: 4625 sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 sleep_before_teardown: 0 subset: 1/8 suite: orch:cephadm suite_branch: quincy-orch-centos-9-teuth suite_path: /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa suite_relpath: qa suite_repo: https://github.com/adk3798/ceph.git suite_sha1: fad5356a94e6044ce1062d748daa5a0f2ed53a7d targets: smithi107.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJO//EBe3Mk7hkhlCPZ7axwzCNsSw3aLHyxfKzdKE4JFhlOBLLJPtbLii43pD39RKDFSZOx4VnjqC1bHK+Ddc8A= smithi162.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQCuQlTnzBqB7dvqJa6O2wR9Ro+rCN+dWRUMBevdpUwvu+2N++4eEoucvkslZ2nUHplXE4unvbC/hXU9x8o5XQ= tasks: - pexec: all: - sudo dnf remove nvme-cli -y - sudo dnf install runc nvmetcli nvme-cli -y - sudo sed -i 's/^#runtime = "crun"/runtime = "runc"/g' /usr/share/containers/containers.conf - sudo sed -i 's/runtime = "crun"/#runtime = "crun"/g' /usr/share/containers/containers.conf - install: null - cephadm: conf: mgr: debug mgr: 20 debug ms: 1 - workunit: clients: client.0: - rados/test.sh - rados/test_pool_quota.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-10_17:53:37 tube: smithi user: gabrioux verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.3401856 2024-09-10T18:36:02.919 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa; will attempt to use it 2024-09-10T18:36:02.920 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_adk3798_ceph_fad5356a94e6044ce1062d748daa5a0f2ed53a7d/qa/tasks 2024-09-10T18:36:02.920 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-10T18:36:02.921 INFO:teuthology.task.internal:Checking packages... 2024-09-10T18:36:02.941 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '5be57395ea235535910512ae73ba8e7ff25fe588' 2024-09-10T18:36:02.941 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-10T18:36:02.941 INFO:teuthology.packaging:ref: None 2024-09-10T18:36:02.941 INFO:teuthology.packaging:tag: None 2024-09-10T18:36:02.941 INFO:teuthology.packaging:branch: wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:36:02.941 INFO:teuthology.packaging:sha1: 5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:36:02.941 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-guits-quincy-2024-09-09-1810-quincy 2024-09-10T18:36:03.095 INFO:teuthology.task.internal:Found packages for ceph version 17.2.7-1606.g5be57395 2024-09-10T18:36:03.096 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-10T18:36:03.102 INFO:teuthology.task.internal:no buildpackages task found 2024-09-10T18:36:03.102 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-10T18:36:03.142 INFO:teuthology.task.internal:Saving configuration 2024-09-10T18:36:03.153 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-10T18:36:03.167 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-10T18:36:03.191 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi107.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:30:59.334989', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJO//EBe3Mk7hkhlCPZ7axwzCNsSw3aLHyxfKzdKE4JFhlOBLLJPtbLii43pD39RKDFSZOx4VnjqC1bHK+Ddc8A='} 2024-09-10T18:36:03.214 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi162.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-10 18:30:59.333685', 'locked_by': 'scheduled_gabrioux@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJQCuQlTnzBqB7dvqJa6O2wR9Ro+rCN+dWRUMBevdpUwvu+2N++4eEoucvkslZ2nUHplXE4unvbC/hXU9x8o5XQ='} 2024-09-10T18:36:03.214 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-10T18:36:03.220 INFO:teuthology.task.internal:roles: ubuntu@smithi107.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'ceph.rgw.foo.a', 'node-exporter.a', 'alertmanager.a'] 2024-09-10T18:36:03.220 INFO:teuthology.task.internal:roles: ubuntu@smithi162.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'grafana.a', 'node-exporter.b', 'ceph.iscsi.iscsi.a'] 2024-09-10T18:36:03.220 INFO:teuthology.run_tasks:Running task console_log... 2024-09-10T18:36:03.295 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f606f993ac0>, signals=[15]) 2024-09-10T18:36:03.295 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-10T18:36:03.300 INFO:teuthology.task.internal:Opening connections... 2024-09-10T18:36:03.301 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi107.front.sepia.ceph.com 2024-09-10T18:36:03.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:36:03.378 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi162.front.sepia.ceph.com 2024-09-10T18:36:03.379 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:36:03.457 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-10T18:36:03.463 DEBUG:teuthology.orchestra.run.smithi107:> uname -m 2024-09-10T18:36:03.482 INFO:teuthology.orchestra.run.smithi107.stdout:x86_64 2024-09-10T18:36:03.482 DEBUG:teuthology.orchestra.run.smithi107:> cat /etc/os-release 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:NAME="CentOS Stream" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION="9" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:ID="centos" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:VERSION_ID="9" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:36:03.541 INFO:teuthology.orchestra.run.smithi107.stdout:ANSI_COLOR="0;31" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:36:03.542 INFO:teuthology.orchestra.run.smithi107.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:36:03.543 INFO:teuthology.lock.ops:Updating smithi107.front.sepia.ceph.com on lock server 2024-09-10T18:36:03.563 DEBUG:teuthology.orchestra.run.smithi162:> uname -m 2024-09-10T18:36:03.580 INFO:teuthology.orchestra.run.smithi162.stdout:x86_64 2024-09-10T18:36:03.580 DEBUG:teuthology.orchestra.run.smithi162:> cat /etc/os-release 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:NAME="CentOS Stream" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION="9" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:ID="centos" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:ID_LIKE="rhel fedora" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:VERSION_ID="9" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:PLATFORM_ID="platform:el9" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:ANSI_COLOR="0;31" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:LOGO="fedora-logo-icon" 2024-09-10T18:36:03.637 INFO:teuthology.orchestra.run.smithi162.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-10T18:36:03.638 INFO:teuthology.orchestra.run.smithi162.stdout:HOME_URL="https://centos.org/" 2024-09-10T18:36:03.638 INFO:teuthology.orchestra.run.smithi162.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-10T18:36:03.638 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-10T18:36:03.638 INFO:teuthology.orchestra.run.smithi162.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-10T18:36:03.638 INFO:teuthology.lock.ops:Updating smithi162.front.sepia.ceph.com on lock server 2024-09-10T18:36:03.663 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-10T18:36:03.670 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-10T18:36:03.692 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-10T18:36:03.693 DEBUG:teuthology.orchestra.run.smithi107:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:36:03.695 DEBUG:teuthology.orchestra.run.smithi162:> test '!' -e /home/ubuntu/cephtest 2024-09-10T18:36:03.711 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-10T18:36:03.716 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-10T18:36:03.716 DEBUG:teuthology.orchestra.run.smithi107:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:36:03.752 DEBUG:teuthology.orchestra.run.smithi162:> test -z $(ls -A /var/lib/ceph) 2024-09-10T18:36:03.779 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-10T18:36:03.831 INFO:teuthology.run_tasks:Running task kernel... 2024-09-10T18:36:03.844 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-09-10T18:36:03.844 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-09-10T18:36:03.845 DEBUG:teuthology.orchestra.run.smithi107:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:36:03.845 DEBUG:teuthology.orchestra.run.smithi162:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-10T18:36:03.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:36:03.861 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-10T18:36:03.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:36:03.863 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-10T18:36:03.919 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:36:03.919 INFO:teuthology.task.kernel:Running kernel on smithi162: 5.14.0-503.el9.x86_64 2024-09-10T18:36:03.919 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum install -y kernel 2024-09-10T18:36:03.920 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-503.el9.x86_64 2024-09-10T18:36:03.920 INFO:teuthology.task.kernel:Running kernel on smithi107: 5.14.0-503.el9.x86_64 2024-09-10T18:36:03.921 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum install -y kernel 2024-09-10T18:36:06.501 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.2 MB 00:00 2024-09-10T18:36:08.865 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.2 MB 00:03 2024-09-10T18:36:11.699 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 12 MB/s | 20 MB 00:01 2024-09-10T18:36:12.134 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 4.5 MB/s | 20 MB 00:04 2024-09-10T18:36:16.249 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 9.2 MB/s | 6.5 MB 00:00 2024-09-10T18:36:16.801 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 6.5 MB 00:00 2024-09-10T18:36:18.008 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-10T18:36:18.243 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 81 kB/s | 19 kB 00:00 2024-09-10T18:36:18.937 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 28 MB/s | 23 MB 00:00 2024-09-10T18:36:19.464 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 21 MB/s | 23 MB 00:01 2024-09-10T18:36:24.384 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-09-10T18:36:24.928 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:36:26.399 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:36:26.399 INFO:teuthology.orchestra.run.smithi107.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:36:26.483 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:36:26.489 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 79 M 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 126 M 2024-09-10T18:36:26.490 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:36:26.627 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 197 kB/s | 23 kB 00:00 2024-09-10T18:36:26.967 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-10T18:36:26.967 INFO:teuthology.orchestra.run.smithi162.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-10T18:36:27.056 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: kernel x86_64 5.14.0-505.el9 baseos 23 k 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core x86_64 5.14.0-505.el9 baseos 16 M 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules x86_64 5.14.0-505.el9 baseos 35 M 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core x86_64 5.14.0-505.el9 baseos 29 M 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:36:27.068 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:36:27.069 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-10T18:36:27.069 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:36:27.069 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 79 M 2024-09-10T18:36:27.069 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 126 M 2024-09-10T18:36:27.069 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:36:27.287 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): kernel-5.14.0-505.el9.x86_64.rpm 222 kB/s | 23 kB 00:00 2024-09-10T18:36:27.879 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 48 MB/s | 29 MB 00:00 2024-09-10T18:36:28.046 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-09-10T18:36:28.480 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 27 MB/s | 35 MB 00:01 2024-09-10T18:36:28.480 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:28.480 INFO:teuthology.orchestra.run.smithi162.stdout:Total 56 MB/s | 79 MB 00:01 2024-09-10T18:36:29.163 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:36:29.531 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:36:29.531 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:36:30.494 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:36:30.494 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:36:32.702 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): kernel-modules-core-5.14.0-505.el9.x86_6 4.7 MB/s | 29 MB 00:06 2024-09-10T18:36:32.961 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:36:33.605 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:36:33.687 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:36:34.143 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): kernel-core-5.14.0-505.el9.x86_64.rpm 2.1 MB/s | 16 MB 00:07 2024-09-10T18:36:36.495 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:36:36.630 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:36:38.234 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): kernel-modules-5.14.0-505.el9.x86_64.rpm 2.9 MB/s | 35 MB 00:11 2024-09-10T18:36:38.235 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:36:38.235 INFO:teuthology.orchestra.run.smithi107.stdout:Total 6.7 MB/s | 79 MB 00:11 2024-09-10T18:36:38.920 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:36:39.265 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:36:40.218 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:36:40.219 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:36:41.476 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:36:41.521 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:36:42.664 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:36:43.558 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-core-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:36:43.634 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:36:46.357 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:36:46.420 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:36:46.553 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:36:51.379 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:36:51.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:36:56.273 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:24.742 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:24.782 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:25.166 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:37:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:37:25.167 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:37:26.062 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:37:26.063 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:37:26.427 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:37:26.518 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.518 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:37:26.518 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:37:26.518 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:37:26.518 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:37:26.518 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-09-10T18:37:26.519 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi162.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:37:26.519 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:37:26.577 INFO:teuthology.orchestra.run.smithi162.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:26.577 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:37:26.577 DEBUG:teuthology.orchestra.run.smithi162:> sudo rpm -qi grub2-tools 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Name : grub2-tools 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Epoch : 1 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Version : 2.06 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Release : 82.el9 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Architecture: x86_64 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:37:26.612 INFO:teuthology.orchestra.run.smithi162.stdout:Group : Unspecified 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Size : 8274670 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:License : GPLv3+ 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Packager : builder@centos.org 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Vendor : CentOS 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Summary : Support tools for GRUB. 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:Description : 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:37:26.613 INFO:teuthology.orchestra.run.smithi162.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:37:26.614 INFO:teuthology.orchestra.run.smithi162.stdout:hardware devices. 2024-09-10T18:37:26.614 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:37:26.614 INFO:teuthology.orchestra.run.smithi162.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:37:26.614 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:37:26.615 INFO:teuthology.task.kernel:Updating grub on smithi162 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:37:26.615 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:37:27.286 INFO:teuthology.orchestra.run.smithi162.stderr:Generating grub configuration file ... 2024-09-10T18:37:28.642 INFO:teuthology.orchestra.run.smithi162.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:37:28.672 INFO:teuthology.orchestra.run.smithi162.stderr:done 2024-09-10T18:37:28.681 DEBUG:teuthology.orchestra.run.smithi162:> mktemp 2024-09-10T18:37:28.698 INFO:teuthology.orchestra.run.smithi162.stdout:/tmp/tmp.AVSKt0HtKy 2024-09-10T18:37:28.698 DEBUG:teuthology.orchestra.run.smithi162:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.AVSKt0HtKy 2024-09-10T18:37:28.765 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0666 /tmp/tmp.AVSKt0HtKy 2024-09-10T18:37:28.907 DEBUG:teuthology.orchestra.remote:smithi162:/tmp/tmp.AVSKt0HtKy is 6KB 2024-09-10T18:37:28.956 DEBUG:teuthology.orchestra.run.smithi162:> rm -fr /tmp/tmp.AVSKt0HtKy 2024-09-10T18:37:28.971 DEBUG:teuthology.orchestra.run.smithi162:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:37:29.034 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:37:29.034 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:37:29.034 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:37:29.034 INFO:teuthology.orchestra.run.smithi162.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:37:29.036 DEBUG:teuthology.orchestra.run.smithi162:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:37:29.179 DEBUG:teuthology.orchestra.run.smithi162:> sudo shutdown -r now 2024-09-10T18:37:34.800 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:34.840 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-modules-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:35.164 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: kernel-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:35.165 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-5.14.0-505.el9.x86_64 1/4 2024-09-10T18:37:35.165 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-core-5.14.0-505.el9.x86_64 2/4 2024-09-10T18:37:35.165 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-5.14.0-505.el9.x86_64 3/4 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : kernel-modules-core-5.14.0-505.el9.x86_64 4/4 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-core-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: kernel-modules-core-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:37:36.032 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:37:36.393 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:37:36.478 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.478 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:37:36.478 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-505.el9.x86_64 2024-09-10T18:37:36.478 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-10T18:37:36.478 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-10T18:37:36.478 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-09-10T18:37:36.479 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi107.front.sepia.ceph.com, path=None, version=distro) 2024-09-10T18:37:36.479 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q kernel | sort -rV | head -n 1 2024-09-10T18:37:36.538 INFO:teuthology.orchestra.run.smithi107.stdout:kernel-5.14.0-505.el9.x86_64 2024-09-10T18:37:36.538 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-505.el9.x86_64 2024-09-10T18:37:36.538 DEBUG:teuthology.orchestra.run.smithi107:> sudo rpm -qi grub2-tools 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Name : grub2-tools 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Epoch : 1 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Version : 2.06 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Release : 82.el9 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Architecture: x86_64 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Group : Unspecified 2024-09-10T18:37:36.570 INFO:teuthology.orchestra.run.smithi107.stdout:Size : 8274670 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:License : GPLv3+ 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Packager : builder@centos.org 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Vendor : CentOS 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Summary : Support tools for GRUB. 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:Description : 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-10T18:37:36.571 INFO:teuthology.orchestra.run.smithi107.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-10T18:37:36.572 INFO:teuthology.orchestra.run.smithi107.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-10T18:37:36.572 INFO:teuthology.orchestra.run.smithi107.stdout:hardware devices. 2024-09-10T18:37:36.572 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:37:36.572 INFO:teuthology.orchestra.run.smithi107.stdout:This subpackage provides tools for support of all platforms. 2024-09-10T18:37:36.573 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-10T18:37:36.573 INFO:teuthology.task.kernel:Updating grub on smithi107 to boot 5.14.0-505.el9.x86_64 2024-09-10T18:37:36.573 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-10T18:37:37.262 INFO:teuthology.orchestra.run.smithi107.stderr:Generating grub configuration file ... 2024-09-10T18:37:38.586 INFO:teuthology.orchestra.run.smithi107.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-10T18:37:38.623 INFO:teuthology.orchestra.run.smithi107.stderr:done 2024-09-10T18:37:38.625 DEBUG:teuthology.orchestra.run.smithi107:> mktemp 2024-09-10T18:37:38.641 INFO:teuthology.orchestra.run.smithi107.stdout:/tmp/tmp.6yctWXqVzV 2024-09-10T18:37:38.642 DEBUG:teuthology.orchestra.run.smithi107:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.6yctWXqVzV 2024-09-10T18:37:38.713 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0666 /tmp/tmp.6yctWXqVzV 2024-09-10T18:37:38.864 DEBUG:teuthology.orchestra.remote:smithi107:/tmp/tmp.6yctWXqVzV is 6KB 2024-09-10T18:37:38.914 DEBUG:teuthology.orchestra.run.smithi107:> rm -fr /tmp/tmp.6yctWXqVzV 2024-09-10T18:37:38.930 DEBUG:teuthology.orchestra.run.smithi107:> sudo /bin/ls /boot/loader/entries || true 2024-09-10T18:37:39.000 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-10T18:37:39.000 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-10T18:37:39.000 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-10T18:37:39.000 INFO:teuthology.orchestra.run.smithi107.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64.conf 2024-09-10T18:37:39.001 DEBUG:teuthology.orchestra.run.smithi107:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-505.el9.x86_64 2024-09-10T18:37:39.166 DEBUG:teuthology.orchestra.run.smithi107:> sudo shutdown -r now 2024-09-10T18:37:59.192 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-09-10T18:37:59.192 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:37:59.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:38:09.177 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-09-10T18:38:09.178 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:38:09.178 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:38:17.661 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-10T18:38:26.671 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:38:26.671 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:38:27.649 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-10T18:38:29.725 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-10T18:38:36.655 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:38:36.656 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:38:39.713 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-10T18:38:41.728 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:38:41.729 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:38:51.741 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:38:51.742 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:39:00.157 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.162 2024-09-10T18:39:10.141 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.107 2024-09-10T18:39:15.164 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:39:15.165 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:39:15.595 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-10T18:39:16.007 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:39:16.007 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:39:16.008 DEBUG:teuthology.orchestra.run.smithi162:> uname -r 2024-09-10T18:39:16.022 INFO:teuthology.orchestra.run.smithi162.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:39:16.022 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:39:16.022 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:39:16.022 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:39:17.023 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-09-10T18:39:17.023 DEBUG:teuthology.orchestra.run.smithi162:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:39:17.109 INFO:teuthology.orchestra.run.smithi162.stdout:ttyS1 2024-09-10T18:39:17.141 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:25.142 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:39:25.143 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:39:25.524 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-10T18:39:26.081 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:39:26.081 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-505.el9.x86_64"... 2024-09-10T18:39:26.082 DEBUG:teuthology.orchestra.run.smithi107:> uname -r 2024-09-10T18:39:26.109 INFO:teuthology.orchestra.run.smithi107.stdout:5.14.0-505.el9.x86_64 2024-09-10T18:39:26.109 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-505.el9.x86_64 vs 5.14.0-505.el9.x86_64 2024-09-10T18:39:26.109 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-10T18:39:26.109 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-10T18:39:27.110 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-09-10T18:39:27.110 DEBUG:teuthology.orchestra.run.smithi107:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-09-10T18:39:27.180 INFO:teuthology.orchestra.run.smithi107.stdout:ttyS1 2024-09-10T18:39:27.216 DEBUG:teuthology.parallel:result is None 2024-09-10T18:39:27.216 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-10T18:39:27.224 INFO:teuthology.task.internal:Creating test directory... 2024-09-10T18:39:27.224 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:39:27.226 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-10T18:39:27.248 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-10T18:39:27.254 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-10T18:39:27.287 INFO:teuthology.task.internal:Creating archive directory... 2024-09-10T18:39:27.287 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:39:27.290 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-10T18:39:27.358 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-10T18:39:27.364 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-10T18:39:27.364 DEBUG:teuthology.orchestra.run.smithi107:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:39:27.367 DEBUG:teuthology.orchestra.run.smithi162:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-10T18:39:27.398 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:39:27.410 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:39:27.413 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:39:27.423 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-10T18:39:27.424 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-10T18:39:27.431 INFO:teuthology.task.internal:Configuring sudo... 2024-09-10T18:39:27.431 DEBUG:teuthology.orchestra.run.smithi107:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:39:27.433 DEBUG:teuthology.orchestra.run.smithi162:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-10T18:39:27.481 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-10T18:39:27.490 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-10T18:39:27.490 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:39:27.499 DEBUG:teuthology.orchestra.run.smithi162:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-10T18:39:27.538 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:39:27.589 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:39:27.658 DEBUG:teuthology.orchestra.run.smithi107:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:39:27.700 DEBUG:teuthology.orchestra.run.smithi107:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:39:27.742 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:39:27.742 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:39:27.807 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:39:27.856 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-10T18:39:27.909 DEBUG:teuthology.orchestra.run.smithi162:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:39:28.000 DEBUG:teuthology.orchestra.run.smithi162:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-10T18:39:28.039 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:39:28.039 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-10T18:39:28.109 DEBUG:teuthology.orchestra.run.smithi107:> sudo service rsyslog restart 2024-09-10T18:39:28.111 DEBUG:teuthology.orchestra.run.smithi162:> sudo service rsyslog restart 2024-09-10T18:39:28.159 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:39:28.207 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T18:39:28.535 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-10T18:39:28.543 INFO:teuthology.task.internal:Starting timer... 2024-09-10T18:39:28.543 INFO:teuthology.run_tasks:Running task pcp... 2024-09-10T18:39:28.582 INFO:teuthology.run_tasks:Running task selinux... 2024-09-10T18:39:28.614 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-10T18:39:28.615 DEBUG:teuthology.orchestra.run.smithi107:> sudo service auditd rotate 2024-09-10T18:39:28.691 INFO:teuthology.orchestra.run.smithi107.stdout:Rotating logs: 2024-09-10T18:39:28.692 DEBUG:teuthology.orchestra.run.smithi162:> sudo service auditd rotate 2024-09-10T18:39:28.762 INFO:teuthology.orchestra.run.smithi162.stdout:Rotating logs: 2024-09-10T18:39:28.764 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-10T18:39:28.764 DEBUG:teuthology.orchestra.run.smithi107:> /usr/sbin/getenforce 2024-09-10T18:39:28.808 INFO:teuthology.orchestra.run.smithi107.stdout:Permissive 2024-09-10T18:39:28.808 DEBUG:teuthology.orchestra.run.smithi162:> /usr/sbin/getenforce 2024-09-10T18:39:28.852 INFO:teuthology.orchestra.run.smithi162.stdout:Permissive 2024-09-10T18:39:28.853 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi107.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi162.front.sepia.ceph.com': 'permissive'} 2024-09-10T18:39:28.853 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:39:28.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:39:28.881 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T18:39:28.909 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:39:28.910 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-10T18:39:28.910 DEBUG:teuthology.orchestra.run.smithi107:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:39:28.953 DEBUG:teuthology.orchestra.run.smithi162:> sudo /usr/sbin/setenforce permissive 2024-09-10T18:39:28.983 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-10T18:39:28.992 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-10T18:39:28.994 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-10T18:39:29.100 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-10T18:39:29.112 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-10T18:39:29.113 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi107.front.sepia.ceph.com,smithi162.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-10T18:43:12.246 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi107.front.sepia.ceph.com'), Remote(name='ubuntu@smithi162.front.sepia.ceph.com')] 2024-09-10T18:43:12.247 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:43:12.249 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi107.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:43:12.332 DEBUG:teuthology.orchestra.run.smithi107:> true 2024-09-10T18:43:12.422 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi107.front.sepia.ceph.com' 2024-09-10T18:43:12.422 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:43:12.423 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi162.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-10T18:43:12.500 DEBUG:teuthology.orchestra.run.smithi162:> true 2024-09-10T18:43:12.580 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi162.front.sepia.ceph.com' 2024-09-10T18:43:12.580 INFO:teuthology.run_tasks:Running task clock... 2024-09-10T18:43:12.591 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-10T18:43:12.592 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:43:12.592 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:43:12.595 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-10T18:43:12.595 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T18:43:12.625 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:43:12.641 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:43:12.649 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-10T18:43:12.664 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-10T18:43:12.666 INFO:teuthology.orchestra.run.smithi107.stderr:sudo: ntpd: command not found 2024-09-10T18:43:12.678 INFO:teuthology.orchestra.run.smithi107.stdout:506 Cannot talk to daemon 2024-09-10T18:43:12.688 INFO:teuthology.orchestra.run.smithi162.stderr:sudo: ntpd: command not found 2024-09-10T18:43:12.693 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:43:12.699 INFO:teuthology.orchestra.run.smithi162.stdout:506 Cannot talk to daemon 2024-09-10T18:43:12.707 INFO:teuthology.orchestra.run.smithi107.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:43:12.713 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-10T18:43:12.727 INFO:teuthology.orchestra.run.smithi162.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-10T18:43:12.766 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.770 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.783 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.787 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T18:43:12.788 INFO:teuthology.run_tasks:Running task pexec... 2024-09-10T18:43:12.797 INFO:teuthology.task.pexec:Executing custom commands... 2024-09-10T18:43:12.797 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi107.front.sepia.ceph.com 2024-09-10T18:43:12.797 DEBUG:teuthology.orchestra.run.smithi107:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:43:12.798 INFO:teuthology.task.pexec:Running commands on host ubuntu@smithi162.front.sepia.ceph.com 2024-09-10T18:43:12.798 DEBUG:teuthology.orchestra.run.smithi162:> TESTDIR=/home/ubuntu/cephtest bash -s 2024-09-10T18:43:13.162 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:Removing: 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:Removing unused dependencies: 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:13.175 INFO:teuthology.orchestra.run.smithi107.stdout:Remove 2 Packages 2024-09-10T18:43:13.176 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:13.176 INFO:teuthology.orchestra.run.smithi107.stdout:Freed space: 5.5 M 2024-09-10T18:43:13.176 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:43:13.177 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:43:13.179 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:43:13.179 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:43:13.192 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:13.192 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-10T18:43:13.192 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Removing: 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout: nvme-cli x86_64 2.9.1-6.el9 @baseos 5.3 M 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Removing unused dependencies: 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout: libnvme x86_64 1.9-3.el9 @baseos 273 k 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Remove 2 Packages 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Freed space: 5.5 M 2024-09-10T18:43:13.193 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:43:13.197 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:43:13.197 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:43:13.249 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:43:13.249 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:43:13.256 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:43:13.257 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:43:13.384 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:43:13.405 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:43:13.449 INFO:teuthology.orchestra.run.smithi107.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:43:13.502 INFO:teuthology.orchestra.run.smithi162.stdout: Erasing : nvme-cli-2.9.1-6.el9.x86_64 1/2 2024-09-10T18:43:13.509 INFO:teuthology.orchestra.run.smithi107.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:43:13.555 INFO:teuthology.orchestra.run.smithi162.stdout: Erasing : libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:43:13.985 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:43:13.985 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:43:14.078 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libnvme-1.9-3.el9.x86_64 2/2 2024-09-10T18:43:14.078 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/2 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout:Removed: 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:14.194 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:43:14.349 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/2 2024-09-10T18:43:14.350 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:Removed: 2024-09-10T18:43:14.350 INFO:teuthology.orchestra.run.smithi162.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:43:14.350 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:14.350 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:43:15.048 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:42:25 PM UTC. 2024-09-10T18:43:15.185 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:43:15.185 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:15.185 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:43:15.185 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-09-10T18:43:15.186 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:15.197 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 5.2 M 2024-09-10T18:43:15.197 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 19 M 2024-09-10T18:43:15.197 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:43:15.209 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:42:26 PM UTC. 2024-09-10T18:43:15.346 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: nvme-cli x86_64 2.9.1-6.el9 baseos 906 k 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: nvmetcli noarch 0.7-3.el9 baseos 44 k 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: runc x86_64 4:1.1.13-4.el9 appstream 3.2 M 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: libnvme x86_64 1.9-3.el9 baseos 106 k 2024-09-10T18:43:15.347 INFO:teuthology.orchestra.run.smithi162.stdout: python3-configshell noarch 1:1.1.30-1.el9 baseos 72 k 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kmod x86_64 0.9-32.el9 baseos 84 k 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urwid x86_64 2.1.2-4.el9 baseos 837 k 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout:Install 7 Packages 2024-09-10T18:43:15.348 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:15.356 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 5.2 M 2024-09-10T18:43:15.357 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 19 M 2024-09-10T18:43:15.357 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:43:15.648 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 360 kB/s | 44 kB 00:00 2024-09-10T18:43:15.654 INFO:teuthology.orchestra.run.smithi162.stdout:(1/7): nvmetcli-0.7-3.el9.noarch.rpm 288 kB/s | 44 kB 00:00 2024-09-10T18:43:15.679 INFO:teuthology.orchestra.run.smithi162.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 590 kB/s | 106 kB 00:00 2024-09-10T18:43:15.690 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): libnvme-1.9-3.el9.x86_64.rpm 640 kB/s | 106 kB 00:00 2024-09-10T18:43:15.724 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-configshell-1.1.30-1.el9.noarch. 956 kB/s | 72 kB 00:00 2024-09-10T18:43:15.729 INFO:teuthology.orchestra.run.smithi162.stdout:(3/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.6 MB/s | 84 kB 00:00 2024-09-10T18:43:15.755 INFO:teuthology.orchestra.run.smithi162.stdout:(4/7): python3-configshell-1.1.30-1.el9.noarch. 713 kB/s | 72 kB 00:00 2024-09-10T18:43:15.765 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): python3-kmod-0.9-32.el9.x86_64.rpm 1.1 MB/s | 84 kB 00:00 2024-09-10T18:43:15.847 INFO:teuthology.orchestra.run.smithi162.stdout:(5/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.5 MB/s | 906 kB 00:00 2024-09-10T18:43:15.874 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.4 MB/s | 837 kB 00:00 2024-09-10T18:43:15.880 INFO:teuthology.orchestra.run.smithi162.stdout:(6/7): python3-urwid-2.1.2-4.el9.x86_64.rpm 5.4 MB/s | 837 kB 00:00 2024-09-10T18:43:15.941 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): nvme-cli-2.9.1-6.el9.x86_64.rpm 2.1 MB/s | 906 kB 00:00 2024-09-10T18:43:16.465 INFO:teuthology.orchestra.run.smithi162.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 4.5 MB/s | 3.2 MB 00:00 2024-09-10T18:43:16.465 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:43:16.465 INFO:teuthology.orchestra.run.smithi162.stdout:Total 4.7 MB/s | 5.2 MB 00:01 2024-09-10T18:43:16.575 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): runc-1.1.13-4.el9.x86_64.rpm 3.9 MB/s | 3.2 MB 00:00 2024-09-10T18:43:16.575 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:43:16.575 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.7 MB/s | 5.2 MB 00:01 2024-09-10T18:43:16.575 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:43:16.590 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:43:16.590 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:43:16.687 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:43:16.701 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:43:16.702 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:43:16.893 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:43:16.894 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:43:17.035 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:43:17.035 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:43:17.369 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:43:17.458 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:43:17.501 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:43:17.504 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:43:17.538 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:43:17.602 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urwid-2.1.2-4.el9.x86_64 1/7 2024-09-10T18:43:17.643 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-configshell-1:1.1.30-1.el9.noarch 2/7 2024-09-10T18:43:17.673 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kmod-0.9-32.el9.x86_64 3/7 2024-09-10T18:43:17.753 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:43:17.786 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:43:17.882 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libnvme-1.9-3.el9.x86_64 4/7 2024-09-10T18:43:17.912 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:43:18.228 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:43:18.255 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:43:18.349 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: nvme-cli-2.9.1-6.el9.x86_64 5/7 2024-09-10T18:43:18.370 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:43:18.382 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:43:18.434 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:18.496 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: nvmetcli-0.7-3.el9.noarch 6/7 2024-09-10T18:43:18.560 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:19.437 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:19.437 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:43:19.437 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:43:19.438 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:43:19.438 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:43:19.438 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:43:19.438 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libnvme-1.9-3.el9.x86_64 1/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvme-cli-2.9.1-6.el9.x86_64 2/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : nvmetcli-0.7-3.el9.noarch 3/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-configshell-1:1.1.30-1.el9.noarch 4/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kmod-0.9-32.el9.x86_64 5/7 2024-09-10T18:43:19.528 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urwid-2.1.2-4.el9.x86_64 6/7 2024-09-10T18:43:19.826 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:19.827 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:43:19.895 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : runc-4:1.1.13-4.el9.x86_64 7/7 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: libnvme-1.9-3.el9.x86_64 nvme-cli-2.9.1-6.el9.x86_64 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: nvmetcli-0.7-3.el9.noarch python3-configshell-1:1.1.30-1.el9.noarch 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kmod-0.9-32.el9.x86_64 python3-urwid-2.1.2-4.el9.x86_64 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: runc-4:1.1.13-4.el9.x86_64 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:19.896 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:43:20.238 DEBUG:teuthology.parallel:result is None 2024-09-10T18:43:20.267 DEBUG:teuthology.parallel:result is None 2024-09-10T18:43:20.267 INFO:teuthology.run_tasks:Running task install... 2024-09-10T18:43:20.276 DEBUG:teuthology.task.install:project ceph 2024-09-10T18:43:20.276 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'}} 2024-09-10T18:43:20.276 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588'} 2024-09-10T18:43:20.276 INFO:teuthology.task.install:Using flavor: default 2024-09-10T18:43:20.281 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-10T18:43:20.282 INFO:teuthology.task.install:extra packages: [] 2024-09-10T18:43:20.282 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:43:20.283 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:43:20.284 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'tag': None, 'wait_for_package': False} 2024-09-10T18:43:20.285 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:43:20.427 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:43:20.427 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:43:20.431 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/ 2024-09-10T18:43:20.432 INFO:teuthology.task.install.rpm:Package version is 17.2.7-1606.g5be57395 2024-09-10T18:43:20.537 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:43:20.537 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:43:20.537 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:43:20.544 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-guits-quincy-2024-09-09-1810-quincy/5be57395ea235535910512ae73ba8e7ff25fe588/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-10T18:43:20.544 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:43:20.544 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-10T18:43:20.563 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:43:20.563 DEBUG:teuthology.orchestra.run.smithi107:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:43:20.571 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-09-10T18:43:20.571 DEBUG:teuthology.orchestra.run.smithi162:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5be57395ea235535910512ae73ba8e7ff25fe588/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-10T18:43:20.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:43:20.637 DEBUG:teuthology.orchestra.run.smithi162:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-10T18:43:20.704 DEBUG:teuthology.orchestra.run.smithi107:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:43:20.713 DEBUG:teuthology.orchestra.run.smithi162:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-10T18:43:20.768 INFO:teuthology.orchestra.run.smithi107.stdout:check_obsoletes = 1 2024-09-10T18:43:20.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum clean all 2024-09-10T18:43:20.776 INFO:teuthology.orchestra.run.smithi162.stdout:check_obsoletes = 1 2024-09-10T18:43:20.778 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum clean all 2024-09-10T18:43:21.006 INFO:teuthology.orchestra.run.smithi162.stdout:45 files removed 2024-09-10T18:43:21.015 INFO:teuthology.orchestra.run.smithi107.stdout:45 files removed 2024-09-10T18:43:21.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-radosgw 2024-09-10T18:43:21.042 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-radosgw 2024-09-10T18:43:21.580 INFO:teuthology.orchestra.run.smithi107.stdout:ceph packages for x86_64 306 kB/s | 77 kB 00:00 2024-09-10T18:43:21.583 INFO:teuthology.orchestra.run.smithi162.stdout:ceph packages for x86_64 298 kB/s | 77 kB 00:00 2024-09-10T18:43:21.847 INFO:teuthology.orchestra.run.smithi107.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-10T18:43:21.850 INFO:teuthology.orchestra.run.smithi162.stdout:ceph noarch packages 48 kB/s | 11 kB 00:00 2024-09-10T18:43:22.106 INFO:teuthology.orchestra.run.smithi107.stdout:ceph source packages 8.1 kB/s | 1.8 kB 00:00 2024-09-10T18:43:22.117 INFO:teuthology.orchestra.run.smithi162.stdout:ceph source packages 7.8 kB/s | 1.8 kB 00:00 2024-09-10T18:43:22.564 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - BaseOS 20 MB/s | 8.2 MB 00:00 2024-09-10T18:43:22.651 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - BaseOS 17 MB/s | 8.2 MB 00:00 2024-09-10T18:43:24.274 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - AppStream 33 MB/s | 20 MB 00:00 2024-09-10T18:43:24.444 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - AppStream 29 MB/s | 20 MB 00:00 2024-09-10T18:43:28.766 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - CRB 9.5 MB/s | 6.5 MB 00:00 2024-09-10T18:43:30.425 INFO:teuthology.orchestra.run.smithi107.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 19 kB 00:00 2024-09-10T18:43:31.204 INFO:teuthology.orchestra.run.smithi107.stdout:Extra Packages for Enterprise Linux 35 MB/s | 23 MB 00:00 2024-09-10T18:43:31.602 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - CRB 1.9 MB/s | 6.5 MB 00:03 2024-09-10T18:43:33.370 INFO:teuthology.orchestra.run.smithi162.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-10T18:43:34.258 INFO:teuthology.orchestra.run.smithi162.stdout:Extra Packages for Enterprise Linux 30 MB/s | 23 MB 00:00 2024-09-10T18:43:36.659 INFO:teuthology.orchestra.run.smithi107.stdout:lab-extras 34 kB/s | 1.7 kB 00:00 2024-09-10T18:43:39.023 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:43:39.024 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:39.024 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrading: 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:43:39.025 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:43:39.026 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout:Install 22 Packages 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout:Upgrade 2 Packages 2024-09-10T18:43:39.027 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:39.028 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 58 M 2024-09-10T18:43:39.028 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:43:39.696 INFO:teuthology.orchestra.run.smithi162.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-09-10T18:43:39.718 INFO:teuthology.orchestra.run.smithi107.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 9.2 MB/s | 5.0 MB 00:00 2024-09-10T18:43:39.777 INFO:teuthology.orchestra.run.smithi107.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 432 kB/s | 25 kB 00:00 2024-09-10T18:43:39.877 INFO:teuthology.orchestra.run.smithi107.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-10T18:43:39.952 INFO:teuthology.orchestra.run.smithi107.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 6.0 MB/s | 458 kB 00:00 2024-09-10T18:43:40.178 INFO:teuthology.orchestra.run.smithi107.stdout:(5/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 14 MB/s | 14 MB 00:00 2024-09-10T18:43:40.236 INFO:teuthology.orchestra.run.smithi107.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 777 kB/s | 45 kB 00:00 2024-09-10T18:43:40.303 INFO:teuthology.orchestra.run.smithi107.stdout:(7/24): python3-ceph-common-17.2.7-1606.g5be573 1.8 MB/s | 120 kB 00:00 2024-09-10T18:43:40.378 INFO:teuthology.orchestra.run.smithi107.stdout:(8/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 12 MB/s | 4.9 MB 00:00 2024-09-10T18:43:40.403 INFO:teuthology.orchestra.run.smithi107.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 1.5 MB/s | 156 kB 00:00 2024-09-10T18:43:40.429 INFO:teuthology.orchestra.run.smithi107.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:43:40.454 INFO:teuthology.orchestra.run.smithi107.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.8 MB/s | 295 kB 00:00 2024-09-10T18:43:40.479 INFO:teuthology.orchestra.run.smithi107.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:43:40.705 INFO:teuthology.orchestra.run.smithi107.stdout:(13/24): ceph-common-17.2.7-1606.g5be57395.el9. 15 MB/s | 22 MB 00:01 2024-09-10T18:43:40.988 INFO:teuthology.orchestra.run.smithi107.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 89 kB/s | 45 kB 00:00 2024-09-10T18:43:41.163 INFO:teuthology.orchestra.run.smithi107.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 226 kB/s | 160 kB 00:00 2024-09-10T18:43:41.280 INFO:teuthology.orchestra.run.smithi107.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 363 kB/s | 42 kB 00:00 2024-09-10T18:43:41.330 INFO:teuthology.orchestra.run.smithi107.stdout:(17/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 6.0 MB/s | 309 kB 00:00 2024-09-10T18:43:41.356 INFO:teuthology.orchestra.run.smithi107.stdout:(18/24): liboath-2.6.7-2.el9.x86_64.rpm 1.9 MB/s | 49 kB 00:00 2024-09-10T18:43:41.381 INFO:teuthology.orchestra.run.smithi107.stdout:(19/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 745 kB/s | 292 kB 00:00 2024-09-10T18:43:41.462 INFO:teuthology.orchestra.run.smithi107.stdout:(20/24): libunwind-1.6.2-1.el9.x86_64.rpm 636 kB/s | 67 kB 00:00 2024-09-10T18:43:41.540 INFO:teuthology.orchestra.run.smithi107.stdout:(21/24): thrift-0.15.0-2.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2024-09-10T18:43:41.674 INFO:teuthology.orchestra.run.smithi107.stdout:(22/24): librdkafka-1.6.1-102.el9.x86_64.rpm 684 kB/s | 662 kB 00:00 2024-09-10T18:43:41.732 INFO:teuthology.orchestra.run.smithi107.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 12 MB/s | 3.1 MB 00:00 2024-09-10T18:43:41.782 INFO:teuthology.orchestra.run.smithi107.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 12 MB/s | 2.9 MB 00:00 2024-09-10T18:43:41.787 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:43:41.787 INFO:teuthology.orchestra.run.smithi107.stdout:Total 21 MB/s | 58 MB 00:02 2024-09-10T18:43:41.853 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:43:41.886 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:43:41.886 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:43:42.097 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:43:42.099 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:42.099 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:43:42.099 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:42.099 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:43:42.099 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 14 M 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrading: 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: librados2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.1 M 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base x86_64 2:17.2.7-1606.g5be57395.el9 ceph 5.0 M 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 22 M 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 637 k 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-10T18:43:42.100 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 458 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2 x86_64 2:17.2.7-1606.g5be57395.el9 ceph 4.9 M 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 45 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common x86_64 2:17.2.7-1606.g5be57395.el9 ceph 120 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs x86_64 2:17.2.7-1606.g5be57395.el9 ceph 156 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados x86_64 2:17.2.7-1606.g5be57395.el9 ceph 320 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 295 k 2024-09-10T18:43:42.101 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw x86_64 2:17.2.7-1606.g5be57395.el9 ceph 99 k 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:Install 22 Packages 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:Upgrade 2 Packages 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 58 M 2024-09-10T18:43:42.102 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:43:42.361 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:43:42.362 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:43:42.823 INFO:teuthology.orchestra.run.smithi162.stdout:(1/24): ceph-base-17.2.7-1606.g5be57395.el9.x86 9.5 MB/s | 5.0 MB 00:00 2024-09-10T18:43:42.881 INFO:teuthology.orchestra.run.smithi162.stdout:(2/24): ceph-selinux-17.2.7-1606.g5be57395.el9. 433 kB/s | 25 kB 00:00 2024-09-10T18:43:42.982 INFO:teuthology.orchestra.run.smithi162.stdout:(3/24): libcephfs2-17.2.7-1606.g5be57395.el9.x8 6.2 MB/s | 637 kB 00:00 2024-09-10T18:43:43.073 INFO:teuthology.orchestra.run.smithi162.stdout:(4/24): libradosstriper1-17.2.7-1606.g5be57395. 4.9 MB/s | 458 kB 00:00 2024-09-10T18:43:43.257 INFO:teuthology.orchestra.run.smithi162.stdout:(5/24): ceph-radosgw-17.2.7-1606.g5be57395.el9. 15 MB/s | 14 MB 00:00 2024-09-10T18:43:43.299 INFO:teuthology.orchestra.run.smithi162.stdout:(6/24): python3-ceph-argparse-17.2.7-1606.g5be5 1.1 MB/s | 45 kB 00:00 2024-09-10T18:43:43.391 INFO:teuthology.orchestra.run.smithi162.stdout:(7/24): librgw2-17.2.7-1606.g5be57395.el9.x86_6 16 MB/s | 4.9 MB 00:00 2024-09-10T18:43:43.416 INFO:teuthology.orchestra.run.smithi162.stdout:(8/24): python3-ceph-common-17.2.7-1606.g5be573 1.0 MB/s | 120 kB 00:00 2024-09-10T18:43:43.441 INFO:teuthology.orchestra.run.smithi162.stdout:(9/24): python3-cephfs-17.2.7-1606.g5be57395.el 3.0 MB/s | 156 kB 00:00 2024-09-10T18:43:43.467 INFO:teuthology.orchestra.run.smithi162.stdout:(10/24): python3-rados-17.2.7-1606.g5be57395.el 6.2 MB/s | 320 kB 00:00 2024-09-10T18:43:43.492 INFO:teuthology.orchestra.run.smithi162.stdout:(11/24): python3-rbd-17.2.7-1606.g5be57395.el9. 5.7 MB/s | 295 kB 00:00 2024-09-10T18:43:43.518 INFO:teuthology.orchestra.run.smithi162.stdout:(12/24): python3-rgw-17.2.7-1606.g5be57395.el9. 1.9 MB/s | 99 kB 00:00 2024-09-10T18:43:43.576 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:43:43.735 INFO:teuthology.orchestra.run.smithi162.stdout:(13/24): ceph-common-17.2.7-1606.g5be57395.el9. 16 MB/s | 22 MB 00:01 2024-09-10T18:43:43.843 INFO:teuthology.orchestra.run.smithi162.stdout:(14/24): librabbitmq-0.11.0-7.el9.x86_64.rpm 139 kB/s | 45 kB 00:00 2024-09-10T18:43:43.877 INFO:teuthology.orchestra.run.smithi162.stdout:(15/24): libpmemobj-1.12.1-1.el9.x86_64.rpm 417 kB/s | 160 kB 00:00 2024-09-10T18:43:43.919 INFO:teuthology.orchestra.run.smithi162.stdout:(16/24): python3-prettytable-0.7.2-27.el9.noarc 1.0 MB/s | 42 kB 00:00 2024-09-10T18:43:43.945 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:43:43.994 INFO:teuthology.orchestra.run.smithi162.stdout:(17/24): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.9 MB/s | 292 kB 00:00 2024-09-10T18:43:44.009 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:43:44.019 INFO:teuthology.orchestra.run.smithi162.stdout:(18/24): gperftools-libs-2.9.1-2.el9.x86_64.rpm 3.0 MB/s | 309 kB 00:00 2024-09-10T18:43:44.045 INFO:teuthology.orchestra.run.smithi162.stdout:(19/24): liboath-2.6.7-2.el9.x86_64.rpm 968 kB/s | 49 kB 00:00 2024-09-10T18:43:44.087 INFO:teuthology.orchestra.run.smithi162.stdout:(20/24): librdkafka-1.6.1-102.el9.x86_64.rpm 1.8 MB/s | 662 kB 00:00 2024-09-10T18:43:44.102 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:43:44.120 INFO:teuthology.orchestra.run.smithi162.stdout:(21/24): libunwind-1.6.2-1.el9.x86_64.rpm 670 kB/s | 67 kB 00:00 2024-09-10T18:43:44.181 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:43:44.187 INFO:teuthology.orchestra.run.smithi162.stdout:(22/24): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-09-10T18:43:44.212 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:43:44.287 INFO:teuthology.orchestra.run.smithi162.stdout:(23/24): librados2-17.2.7-1606.g5be57395.el9.x8 16 MB/s | 3.1 MB 00:00 2024-09-10T18:43:44.301 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:43:44.346 INFO:teuthology.orchestra.run.smithi162.stdout:(24/24): librbd1-17.2.7-1606.g5be57395.el9.x86_ 13 MB/s | 2.9 MB 00:00 2024-09-10T18:43:44.347 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:43:44.351 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:43:44.351 INFO:teuthology.orchestra.run.smithi162.stdout:Total 26 MB/s | 58 MB 00:02 2024-09-10T18:43:44.376 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:43:44.412 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:43:44.444 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:43:44.445 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:43:44.813 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:43:44.841 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:43:44.903 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:43:44.924 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:43:44.925 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:43:44.932 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:43:44.977 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:43:45.012 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:43:45.074 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:43:45.102 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:43:45.167 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:43:45.209 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:43:45.252 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:43:45.285 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:43:45.586 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:43:45.631 INFO:teuthology.orchestra.run.smithi107.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:43:45.753 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:43:45.828 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:43:45.858 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:43:46.114 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:43:46.454 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/26 2024-09-10T18:43:46.557 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:43:46.649 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:43:46.736 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:43:46.767 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:43:46.855 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:43:46.910 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : thrift-0.15.0-2.el9.x86_64 5/26 2024-09-10T18:43:46.939 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/26 2024-09-10T18:43:47.385 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/26 2024-09-10T18:43:47.412 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:43:47.458 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 8/26 2024-09-10T18:43:47.495 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : liboath-2.6.7-2.el9.x86_64 9/26 2024-09-10T18:43:47.538 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 10/26 2024-09-10T18:43:47.575 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:43:47.637 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:43:47.665 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:43:47.731 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 13/26 2024-09-10T18:43:47.777 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 14/26 2024-09-10T18:43:47.821 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 15/26 2024-09-10T18:43:47.853 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 16/26 2024-09-10T18:43:48.151 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 17/26 2024-09-10T18:43:48.200 INFO:teuthology.orchestra.run.smithi162.stdout: Upgrading : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:43:48.312 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 18/26 2024-09-10T18:43:48.369 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:48.381 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 19/26 2024-09-10T18:43:48.401 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:48.411 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 20/26 2024-09-10T18:43:48.942 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:48.980 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:43:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:43:49.084 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:43:49.084 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:43:49.084 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:43:49.124 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:43:50.890 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:50.924 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:51.460 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 21/26 2024-09-10T18:43:51.503 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:43:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 22/26 2024-09-10T18:43:51.601 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-10T18:43:51.601 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-10T18:43:51.601 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:43:51.638 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:43:58.987 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:43:58.987 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /sys 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /proc 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /mnt 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /var/tmp 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /home 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /root 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout:skipping the directory /tmp 2024-09-10T18:43:58.988 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:00.280 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:44:00.360 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:44:00.361 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:00.361 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:44:00.361 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:44:00.361 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:44:00.361 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:00.362 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:44:00.450 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:44:00.450 INFO:teuthology.orchestra.run.smithi107.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:01.490 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:44:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /sys 2024-09-10T18:44:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /proc 2024-09-10T18:44:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /mnt 2024-09-10T18:44:01.490 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /var/tmp 2024-09-10T18:44:01.491 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /home 2024-09-10T18:44:01.491 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /root 2024-09-10T18:44:01.491 INFO:teuthology.orchestra.run.smithi162.stdout:skipping the directory /tmp 2024-09-10T18:44:01.491 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:01.768 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:44:01.769 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:44:01.770 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout:Upgraded: 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.542 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.543 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:02.544 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:02.771 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:44:02.853 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 24/26 2024-09-10T18:44:02.856 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:02.856 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-10T18:44:02.856 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:44:02.856 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-10T18:44:02.856 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:02.857 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:44:02.897 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-test 2024-09-10T18:44:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 25/26 2024-09-10T18:44:02.949 INFO:teuthology.orchestra.run.smithi162.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:03.463 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:03.596 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:03.596 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:03.596 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:03.596 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:03.597 INFO:teuthology.orchestra.run.smithi107.stdout:Install 4 Packages 2024-09-10T18:44:03.598 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:03.598 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 48 M 2024-09-10T18:44:03.598 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 193 M 2024-09-10T18:44:03.598 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:03.840 INFO:teuthology.orchestra.run.smithi107.stdout:(1/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 740 kB/s | 163 kB 00:00 2024-09-10T18:44:04.207 INFO:teuthology.orchestra.run.smithi107.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 174 kB/s | 64 kB 00:00 2024-09-10T18:44:04.249 INFO:teuthology.orchestra.run.smithi107.stdout:(3/4): socat-1.7.4.1-6.el9.x86_64.rpm 484 kB/s | 304 kB 00:00 2024-09-10T18:44:04.263 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 1/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 3/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 4/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 5/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_6 6/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 7/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9. 8/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x8 9/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 10/26 2024-09-10T18:44:04.264 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 11/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 12/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 13/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 14/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 15/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 16/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 17/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 18/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 19/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 20/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 21/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 22/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:17.2.7-1606.g5be57395.el9.x86_64 23/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 24/26 2024-09-10T18:44:04.265 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 25/26 2024-09-10T18:44:05.124 INFO:teuthology.orchestra.run.smithi107.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 32 MB/s | 48 MB 00:01 2024-09-10T18:44:05.126 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:05.126 INFO:teuthology.orchestra.run.smithi107.stdout:Total 32 MB/s | 48 MB 00:01 2024-09-10T18:44:05.154 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:05.173 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:05.173 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 26/26 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout:Upgraded: 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: librados2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-radosgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-selinux-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.209 INFO:teuthology.orchestra.run.smithi162.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: libradosstriper1-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: librgw2-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-argparse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ceph-common-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.210 INFO:teuthology.orchestra.run.smithi162.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:05.211 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:05.538 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:05.538 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:05.613 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-test 2024-09-10T18:44:06.088 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:06.170 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:44:06.185 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:27 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:06.214 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:44:06.251 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:44:06.318 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:06.319 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:06.319 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:06.319 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:06.319 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test x86_64 2:17.2.7-1606.g5be57395.el9 ceph 48 M 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite x86_64 2:17.2.7-1606.g5be57395.el9 ceph 163 k 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout:Install 4 Packages 2024-09-10T18:44:06.320 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:06.321 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 48 M 2024-09-10T18:44:06.321 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 193 M 2024-09-10T18:44:06.321 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:06.499 INFO:teuthology.orchestra.run.smithi162.stdout:(1/4): socat-1.7.4.1-6.el9.x86_64.rpm 1.9 MB/s | 304 kB 00:00 2024-09-10T18:44:06.532 INFO:teuthology.orchestra.run.smithi162.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.9 MB/s | 64 kB 00:00 2024-09-10T18:44:06.582 INFO:teuthology.orchestra.run.smithi162.stdout:(3/4): libcephsqlite-17.2.7-1606.g5be57395.el9. 680 kB/s | 163 kB 00:00 2024-09-10T18:44:08.166 INFO:teuthology.orchestra.run.smithi162.stdout:(4/4): ceph-test-17.2.7-1606.g5be57395.el9.x86_ 26 MB/s | 48 MB 00:01 2024-09-10T18:44:08.168 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:08.168 INFO:teuthology.orchestra.run.smithi162.stdout:Total 26 MB/s | 48 MB 00:01 2024-09-10T18:44:08.196 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:08.215 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:08.215 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:08.579 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:08.579 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:09.133 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:09.216 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-09-10T18:44:09.256 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 2/4 2024-09-10T18:44:09.276 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:44:10.642 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:44:10.724 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:44:12.967 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:44:12.968 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:44:12.968 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:44:12.968 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:13.283 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:13.563 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph 2024-09-10T18:44:13.687 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 3/4 2024-09-10T18:44:13.762 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:44:14.134 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:14.270 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:44:14.273 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:44:14.274 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:44:14.275 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout:Install 34 Packages 2024-09-10T18:44:14.276 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:14.277 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 27 M 2024-09-10T18:44:14.277 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 96 M 2024-09-10T18:44:14.277 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:14.621 INFO:teuthology.orchestra.run.smithi107.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 47 kB/s | 6.5 kB 00:00 2024-09-10T18:44:14.804 INFO:teuthology.orchestra.run.smithi107.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.4 MB/s | 1.4 MB 00:00 2024-09-10T18:44:14.846 INFO:teuthology.orchestra.run.smithi107.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-10T18:44:14.997 INFO:teuthology.orchestra.run.smithi107.stdout:(4/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 10 MB/s | 3.9 MB 00:00 2024-09-10T18:44:15.022 INFO:teuthology.orchestra.run.smithi107.stdout:(5/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 1.3 MB/s | 240 kB 00:00 2024-09-10T18:44:15.322 INFO:teuthology.orchestra.run.smithi107.stdout:(6/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 30 MB/s | 15 MB 00:00 2024-09-10T18:44:15.747 INFO:teuthology.orchestra.run.smithi107.stdout:(7/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 349 kB/s | 253 kB 00:00 2024-09-10T18:44:15.781 INFO:teuthology.orchestra.run.smithi107.stdout:(8/34): python3-ply-3.11-14.el9.noarch.rpm 232 kB/s | 106 kB 00:00 2024-09-10T18:44:15.864 INFO:teuthology.orchestra.run.smithi107.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 1.1 MB/s | 135 kB 00:00 2024-09-10T18:44:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 4/4 2024-09-10T18:44:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 1/4 2024-09-10T18:44:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2/4 2024-09-10T18:44:15.973 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 3/4 2024-09-10T18:44:15.974 INFO:teuthology.orchestra.run.smithi107.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-09-10T18:44:15.998 INFO:teuthology.orchestra.run.smithi107.stdout:(11/34): python3-requests-2.25.1-8.el9.noarch.r 574 kB/s | 125 kB 00:00 2024-09-10T18:44:16.032 INFO:teuthology.orchestra.run.smithi107.stdout:(12/34): python3-urllib3-1.26.5-6.el9.noarch.rp 1.3 MB/s | 215 kB 00:00 2024-09-10T18:44:16.090 INFO:teuthology.orchestra.run.smithi107.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 379 kB/s | 35 kB 00:00 2024-09-10T18:44:16.132 INFO:teuthology.orchestra.run.smithi107.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-10T18:44:16.158 INFO:teuthology.orchestra.run.smithi107.stdout:(15/34): python3-packaging-20.9-5.el9.noarch.rp 614 kB/s | 77 kB 00:00 2024-09-10T18:44:16.183 INFO:teuthology.orchestra.run.smithi107.stdout:(16/34): python3-pytz-2021.1-5.el9.noarch.rpm 552 kB/s | 51 kB 00:00 2024-09-10T18:44:16.209 INFO:teuthology.orchestra.run.smithi107.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 549 kB/s | 42 kB 00:00 2024-09-10T18:44:16.259 INFO:teuthology.orchestra.run.smithi107.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 431 kB/s | 43 kB 00:00 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-test-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: libcephsqlite-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:16.277 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:16.284 INFO:teuthology.orchestra.run.smithi107.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 464 kB/s | 11 kB 00:00 2024-09-10T18:44:16.309 INFO:teuthology.orchestra.run.smithi107.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-09-10T18:44:16.326 INFO:teuthology.orchestra.run.smithi107.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 423 kB/s | 18 kB 00:00 2024-09-10T18:44:16.352 INFO:teuthology.orchestra.run.smithi107.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 552 kB/s | 23 kB 00:00 2024-09-10T18:44:16.377 INFO:teuthology.orchestra.run.smithi107.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-09-10T18:44:16.402 INFO:teuthology.orchestra.run.smithi107.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 391 kB/s | 20 kB 00:00 2024-09-10T18:44:16.428 INFO:teuthology.orchestra.run.smithi107.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 920 kB/s | 46 kB 00:00 2024-09-10T18:44:16.453 INFO:teuthology.orchestra.run.smithi107.stdout:(26/34): python3-more-itertools-8.12.0-2.el9.no 1.5 MB/s | 79 kB 00:00 2024-09-10T18:44:16.478 INFO:teuthology.orchestra.run.smithi107.stdout:(27/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-10T18:44:16.495 INFO:teuthology.orchestra.run.smithi107.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 391 kB/s | 16 kB 00:00 2024-09-10T18:44:16.520 INFO:teuthology.orchestra.run.smithi107.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.1 MB/s | 90 kB 00:00 2024-09-10T18:44:16.538 INFO:teuthology.orchestra.run.smithi107.stdout:(30/34): python3-tempora-5.0.0-2.el9.noarch.rpm 854 kB/s | 36 kB 00:00 2024-09-10T18:44:16.563 INFO:teuthology.orchestra.run.smithi107.stdout:(31/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-10T18:44:16.588 INFO:teuthology.orchestra.run.smithi107.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 806 kB/s | 20 kB 00:00 2024-09-10T18:44:16.596 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph 2024-09-10T18:44:16.613 INFO:teuthology.orchestra.run.smithi107.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-10T18:44:16.638 INFO:teuthology.orchestra.run.smithi107.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-10T18:44:16.643 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:16.643 INFO:teuthology.orchestra.run.smithi107.stdout:Total 11 MB/s | 27 MB 00:02 2024-09-10T18:44:16.775 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:16.812 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:16.812 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:17.172 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:38 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:17.184 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:17.184 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:17.307 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:17.309 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph x86_64 2:17.2.7-1606.g5be57395.el9 ceph 6.5 k 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.1 M 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr x86_64 2:17.2.7-1606.g5be57395.el9 ceph 1.4 M 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 240 k 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon x86_64 2:17.2.7-1606.g5be57395.el9 ceph 3.9 M 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 15 M 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-10T18:44:17.310 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-10T18:44:17.311 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-10T18:44:17.312 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout:Install 34 Packages 2024-09-10T18:44:17.313 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:17.314 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 27 M 2024-09-10T18:44:17.314 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 96 M 2024-09-10T18:44:17.314 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:17.905 INFO:teuthology.orchestra.run.smithi162.stdout:(1/34): ceph-17.2.7-1606.g5be57395.el9.x86_64.r 48 kB/s | 6.5 kB 00:00 2024-09-10T18:44:18.081 INFO:teuthology.orchestra.run.smithi162.stdout:(2/34): ceph-mgr-17.2.7-1606.g5be57395.el9.x86_ 4.5 MB/s | 1.4 MB 00:00 2024-09-10T18:44:18.131 INFO:teuthology.orchestra.run.smithi162.stdout:(3/34): ceph-mds-17.2.7-1606.g5be57395.el9.x86_ 5.7 MB/s | 2.1 MB 00:00 2024-09-10T18:44:18.198 INFO:teuthology.orchestra.run.smithi162.stdout:(4/34): ceph-mgr-modules-core-17.2.7-1606.g5be5 3.5 MB/s | 240 kB 00:00 2024-09-10T18:44:18.265 INFO:teuthology.orchestra.run.smithi162.stdout:(5/34): ceph-mon-17.2.7-1606.g5be57395.el9.x86_ 11 MB/s | 3.9 MB 00:00 2024-09-10T18:44:18.398 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:18.415 INFO:teuthology.orchestra.run.smithi162.stdout:(6/34): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-09-10T18:44:18.458 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:44:18.466 INFO:teuthology.orchestra.run.smithi162.stdout:(7/34): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-10T18:44:18.490 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:44:18.523 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:44:18.615 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:44:18.616 INFO:teuthology.orchestra.run.smithi162.stdout:(8/34): ceph-osd-17.2.7-1606.g5be57395.el9.x86_ 29 MB/s | 15 MB 00:00 2024-09-10T18:44:18.641 INFO:teuthology.orchestra.run.smithi162.stdout:(9/34): python3-pycparser-2.20-6.el9.noarch.rpm 772 kB/s | 135 kB 00:00 2024-09-10T18:44:18.675 INFO:teuthology.orchestra.run.smithi162.stdout:(10/34): python3-cryptography-36.0.1-4.el9.x86_ 3.0 MB/s | 1.2 MB 00:00 2024-09-10T18:44:18.705 INFO:teuthology.orchestra.run.smithi162.stdout:(11/34): python3-urllib3-1.26.5-6.el9.noarch.rp 3.3 MB/s | 215 kB 00:00 2024-09-10T18:44:18.709 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:44:18.751 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:44:18.764 INFO:teuthology.orchestra.run.smithi162.stdout:(12/34): python3-requests-2.25.1-8.el9.noarch.r 845 kB/s | 125 kB 00:00 2024-09-10T18:44:18.797 INFO:teuthology.orchestra.run.smithi162.stdout:(13/34): python3-markupsafe-1.1.1-12.el9.x86_64 378 kB/s | 35 kB 00:00 2024-09-10T18:44:18.798 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:44:18.823 INFO:teuthology.orchestra.run.smithi162.stdout:(14/34): python3-mako-1.1.4-6.el9.noarch.rpm 1.1 MB/s | 172 kB 00:00 2024-09-10T18:44:18.829 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:44:18.848 INFO:teuthology.orchestra.run.smithi162.stdout:(15/34): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-10T18:44:18.863 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:44:18.874 INFO:teuthology.orchestra.run.smithi162.stdout:(16/34): python3-packaging-20.9-5.el9.noarch.rp 704 kB/s | 77 kB 00:00 2024-09-10T18:44:18.896 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:44:18.899 INFO:teuthology.orchestra.run.smithi162.stdout:(17/34): python3-toml-0.10.2-6.el9.noarch.rpm 549 kB/s | 42 kB 00:00 2024-09-10T18:44:18.937 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:44:18.949 INFO:teuthology.orchestra.run.smithi162.stdout:(18/34): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 431 kB/s | 43 kB 00:00 2024-09-10T18:44:18.972 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:44:18.974 INFO:teuthology.orchestra.run.smithi162.stdout:(19/34): python3-jaraco-8.2.1-3.el9.noarch.rpm 460 kB/s | 11 kB 00:00 2024-09-10T18:44:19.000 INFO:teuthology.orchestra.run.smithi162.stdout:(20/34): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.7 MB/s | 172 kB 00:00 2024-09-10T18:44:19.006 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:44:19.025 INFO:teuthology.orchestra.run.smithi162.stdout:(21/34): python3-jaraco-classes-3.2.1-5.el9.noa 352 kB/s | 18 kB 00:00 2024-09-10T18:44:19.051 INFO:teuthology.orchestra.run.smithi162.stdout:(22/34): python3-jaraco-collections-3.0.0-8.el9 925 kB/s | 23 kB 00:00 2024-09-10T18:44:19.076 INFO:teuthology.orchestra.run.smithi162.stdout:(23/34): python3-jaraco-functools-3.5.0-2.el9.n 386 kB/s | 19 kB 00:00 2024-09-10T18:44:19.076 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:44:19.101 INFO:teuthology.orchestra.run.smithi162.stdout:(24/34): python3-jaraco-text-3.2.0-6.el9.noarch 390 kB/s | 20 kB 00:00 2024-09-10T18:44:19.118 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:44:19.126 INFO:teuthology.orchestra.run.smithi162.stdout:(25/34): python3-logutils-0.3.5-21.el9.noarch.r 918 kB/s | 46 kB 00:00 2024-09-10T18:44:19.152 INFO:teuthology.orchestra.run.smithi162.stdout:(26/34): python3-cherrypy-18.6.1-2.el9.noarch.r 1.4 MB/s | 358 kB 00:00 2024-09-10T18:44:19.177 INFO:teuthology.orchestra.run.smithi162.stdout:(27/34): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-09-10T18:44:19.193 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:44:19.202 INFO:teuthology.orchestra.run.smithi162.stdout:(28/34): python3-portend-3.1.0-2.el9.noarch.rpm 656 kB/s | 16 kB 00:00 2024-09-10T18:44:19.227 INFO:teuthology.orchestra.run.smithi162.stdout:(29/34): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.7 MB/s | 90 kB 00:00 2024-09-10T18:44:19.261 INFO:teuthology.orchestra.run.smithi162.stdout:(30/34): python3-pecan-1.4.2-3.el9.noarch.rpm 2.0 MB/s | 272 kB 00:00 2024-09-10T18:44:19.286 INFO:teuthology.orchestra.run.smithi162.stdout:(31/34): python3-tempora-5.0.0-2.el9.noarch.rpm 425 kB/s | 36 kB 00:00 2024-09-10T18:44:19.312 INFO:teuthology.orchestra.run.smithi162.stdout:(32/34): python3-zc-lockfile-2.0-10.el9.noarch. 802 kB/s | 20 kB 00:00 2024-09-10T18:44:19.331 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:44:19.337 INFO:teuthology.orchestra.run.smithi162.stdout:(33/34): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-10T18:44:19.362 INFO:teuthology.orchestra.run.smithi162.stdout:(34/34): python3-werkzeug-2.0.3-3.el9.1.noarch. 4.1 MB/s | 427 kB 00:00 2024-09-10T18:44:19.366 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:19.366 INFO:teuthology.orchestra.run.smithi162.stdout:Total 13 MB/s | 27 MB 00:02 2024-09-10T18:44:19.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:44:19.465 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:44:19.489 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:19.512 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:44:19.526 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:19.526 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:19.561 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:44:19.616 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:44:19.800 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:44:19.849 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:44:19.910 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:44:19.919 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:19.919 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:19.999 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:44:20.045 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:44:20.164 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:44:20.311 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:44:20.343 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:44:20.368 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:44:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:44:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:44:20.369 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:44:20.369 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:21.177 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:21.249 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/34 2024-09-10T18:44:21.289 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/34 2024-09-10T18:44:21.322 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 3/34 2024-09-10T18:44:21.423 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 4/34 2024-09-10T18:44:21.517 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 5/34 2024-09-10T18:44:21.559 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 6/34 2024-09-10T18:44:21.602 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 7/34 2024-09-10T18:44:21.632 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 8/34 2024-09-10T18:44:21.667 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 9/34 2024-09-10T18:44:21.700 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 10/34 2024-09-10T18:44:21.728 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:44:21.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 11/34 2024-09-10T18:44:21.756 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:44:21.756 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:21.756 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:44:21.757 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:44:21.757 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:44:21.757 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:21.784 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 12/34 2024-09-10T18:44:21.818 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 13/34 2024-09-10T18:44:21.875 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 14/34 2024-09-10T18:44:21.921 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:44:21.998 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 16/34 2024-09-10T18:44:22.138 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 17/34 2024-09-10T18:44:22.145 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:44:22.170 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:22.215 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 18/34 2024-09-10T18:44:22.268 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 19/34 2024-09-10T18:44:22.316 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 20/34 2024-09-10T18:44:22.365 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-ply-3.11-14.el9.noarch 21/34 2024-09-10T18:44:22.396 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:44:22.420 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:22.421 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 22/34 2024-09-10T18:44:22.484 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:44:22.606 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 23/34 2024-09-10T18:44:22.662 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 24/34 2024-09-10T18:44:22.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 25/34 2024-09-10T18:44:22.837 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 26/34 2024-09-10T18:44:22.874 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 27/34 2024-09-10T18:44:22.993 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 28/34 2024-09-10T18:44:23.141 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 29/34 2024-09-10T18:44:23.172 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 30/34 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-10T18:44:23.198 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:44:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:44:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:44:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:44:24.126 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:44:24.127 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:44:24.128 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:44:24.129 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:44:24.129 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:44:24.129 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:44:24.129 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:44:24.566 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 31/34 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-10T18:44:24.594 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:24.972 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:44:24.972 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:24.972 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:24.972 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:44:24.973 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:44:24.974 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:24.975 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:44:24.999 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 32/34 2024-09-10T18:44:24.999 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:25.000 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-10T18:44:25.000 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:44:25.000 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-10T18:44:25.000 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:25.201 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:44:25.224 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 33/34 2024-09-10T18:44:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-10T18:44:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:44:25.225 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-10T18:44:25.225 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:25.289 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:44:25.326 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-base 2024-09-10T18:44:25.898 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:25.988 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:44:26.029 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:26.030 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:44:26.030 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:26.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install cephadm 2024-09-10T18:44:26.660 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:26.791 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:26.792 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:26.792 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:26.792 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:26.792 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 83 k 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 375 k 2024-09-10T18:44:26.793 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:26.986 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 429 kB/s | 83 kB 00:00 2024-09-10T18:44:26.987 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:26.987 INFO:teuthology.orchestra.run.smithi107.stdout:Total 425 kB/s | 83 kB 00:00 2024-09-10T18:44:26.987 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:26.992 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:26.992 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:26.994 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:26.995 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:27.062 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 34/34 2024-09-10T18:44:27.062 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-2:17.2.7-1606.g5be57395.el9.x86_64 1/34 2024-09-10T18:44:27.062 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 3/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 4/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 5/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9. 6/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/34 2024-09-10T18:44:27.063 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 12/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 13/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 14/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 15/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 18/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 19/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 20/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 21/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 22/34 2024-09-10T18:44:27.064 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 23/34 2024-09-10T18:44:27.065 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 24/34 2024-09-10T18:44:27.065 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 25/34 2024-09-10T18:44:27.065 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:27.066 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 26/34 2024-09-10T18:44:27.066 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 27/34 2024-09-10T18:44:27.066 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 28/34 2024-09-10T18:44:27.066 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 29/34 2024-09-10T18:44:27.067 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 30/34 2024-09-10T18:44:27.067 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 31/34 2024-09-10T18:44:27.067 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 32/34 2024-09-10T18:44:27.067 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 33/34 2024-09-10T18:44:27.495 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:27.592 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:27.910 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 34/34 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mds-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:27.977 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-modules-core-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mon-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-osd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-10T18:44:27.978 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-ply-3.11-14.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-10T18:44:27.979 INFO:teuthology.orchestra.run.smithi162.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:27.980 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:28.155 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:28.155 INFO:teuthology.orchestra.run.smithi107.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:28.156 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:28.156 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:28.369 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:44:28.370 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-base 2024-09-10T18:44:28.937 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:49 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:28.952 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:52 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:29.026 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-base-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:44:29.067 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:29.068 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:44:29.068 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:29.091 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:44:29.092 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 137 k 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 379 k 2024-09-10T18:44:29.093 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:29.149 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install cephadm 2024-09-10T18:44:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 656 kB/s | 137 kB 00:00 2024-09-10T18:44:29.303 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:29.304 INFO:teuthology.orchestra.run.smithi107.stdout:Total 651 kB/s | 137 kB 00:00 2024-09-10T18:44:29.304 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:29.311 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:29.311 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:29.374 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:29.374 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:29.537 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:29.604 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:29.630 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:29.630 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:29.630 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:44:29.630 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:29.711 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:50 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:29.843 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 83 k 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:44:29.844 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:29.845 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 83 k 2024-09-10T18:44:29.845 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 375 k 2024-09-10T18:44:29.845 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:30.085 INFO:teuthology.orchestra.run.smithi162.stdout:cephadm-17.2.7-1606.g5be57395.el9.noarch.rpm 345 kB/s | 83 kB 00:00 2024-09-10T18:44:30.085 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:30.085 INFO:teuthology.orchestra.run.smithi162.stdout:Total 342 kB/s | 83 kB 00:00 2024-09-10T18:44:30.086 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:30.091 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:30.091 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:30.093 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:30.093 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:30.169 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:30.545 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:30.545 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:30.546 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:30.546 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:30.546 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:30.546 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:30.643 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:30.746 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:30.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr 2024-09-10T18:44:31.116 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:31.361 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:31.365 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:44:31.366 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:31.366 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:31.366 INFO:teuthology.orchestra.run.smithi162.stdout: cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:31.366 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:31.366 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:31.455 INFO:teuthology.orchestra.run.smithi107.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:44:31.498 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:31.499 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:44:31.499 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:31.568 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:44:31.576 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-immutable-object-cache 2024-09-10T18:44:32.134 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:32.147 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:53 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:32.267 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:32.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:32.268 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:32.268 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout:Installing weak dependencies: 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:44:32.269 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:32.270 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:32.270 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:32.270 INFO:teuthology.orchestra.run.smithi107.stdout:Install 7 Packages 2024-09-10T18:44:32.270 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:32.270 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 3.7 M 2024-09-10T18:44:32.271 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 80 M 2024-09-10T18:44:32.271 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:32.283 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache x86_64 2:17.2.7-1606.g5be57395.el9 ceph 137 k 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:32.284 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:32.285 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:44:32.285 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:32.285 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 137 k 2024-09-10T18:44:32.285 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 379 k 2024-09-10T18:44:32.285 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:32.484 INFO:teuthology.orchestra.run.smithi107.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 160 kB/s | 24 kB 00:00 2024-09-10T18:44:32.486 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-immutable-object-cache-17.2.7-1606.g5be573 684 kB/s | 137 kB 00:00 2024-09-10T18:44:32.486 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:32.487 INFO:teuthology.orchestra.run.smithi162.stdout:Total 679 kB/s | 137 kB 00:00 2024-09-10T18:44:32.487 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:32.494 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:32.494 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:32.509 INFO:teuthology.orchestra.run.smithi107.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 82 kB/s | 15 kB 00:00 2024-09-10T18:44:32.551 INFO:teuthology.orchestra.run.smithi107.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:44:32.557 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:32.557 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:32.576 INFO:teuthology.orchestra.run.smithi107.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:44:32.602 INFO:teuthology.orchestra.run.smithi107.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 443 kB/s | 41 kB 00:00 2024-09-10T18:44:32.627 INFO:teuthology.orchestra.run.smithi107.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:44:32.707 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:32.752 INFO:teuthology.orchestra.run.smithi107.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.0 MB/s | 3.4 MB 00:00 2024-09-10T18:44:32.756 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:32.756 INFO:teuthology.orchestra.run.smithi107.stdout:Total 7.5 MB/s | 3.7 MB 00:00 2024-09-10T18:44:32.756 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:32.764 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:32.764 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:32.770 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:32.795 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:32.796 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:44:32.796 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-10T18:44:32.796 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:32.851 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:32.851 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:33.166 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:33.221 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:44:33.254 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:44:33.300 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:44:33.333 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:44:33.362 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-immutable-object-cache-2:17.2.7-1606.g5be57395. 1/1 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-immutable-object-cache-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:33.742 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:33.964 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr 2024-09-10T18:44:34.348 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:44:34.382 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:44:34.539 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:55 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:34.633 INFO:teuthology.orchestra.run.smithi162.stdout:Package ceph-mgr-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:44:34.676 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:34.677 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:44:34.677 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:34.743 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-dashboard 2024-09-10T18:44:35.314 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:00:56 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:35.362 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:44:35.363 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:44:35.447 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:35.447 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 3.4 M 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 24 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 15 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt noarch 2.4.0-1.el9 epel 41 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout:Installing weak dependencies: 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto noarch 2.4.0-1.el9 epel 9.0 k 2024-09-10T18:44:35.448 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:Install 7 Packages 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 3.7 M 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 80 M 2024-09-10T18:44:35.449 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:35.636 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:35.637 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:35.640 INFO:teuthology.orchestra.run.smithi162.stdout:(1/7): ceph-grafana-dashboards-17.2.7-1606.g5be 154 kB/s | 24 kB 00:00 2024-09-10T18:44:35.665 INFO:teuthology.orchestra.run.smithi162.stdout:(2/7): ceph-prometheus-alerts-17.2.7-1606.g5be5 80 kB/s | 15 kB 00:00 2024-09-10T18:44:35.708 INFO:teuthology.orchestra.run.smithi162.stdout:(3/7): python3-jwt+crypto-2.4.0-1.el9.noarch.rp 135 kB/s | 9.0 kB 00:00 2024-09-10T18:44:35.733 INFO:teuthology.orchestra.run.smithi162.stdout:(4/7): python3-repoze-lru-0.7-16.el9.noarch.rpm 1.3 MB/s | 31 kB 00:00 2024-09-10T18:44:35.758 INFO:teuthology.orchestra.run.smithi162.stdout:(5/7): python3-jwt-2.4.0-1.el9.noarch.rpm 441 kB/s | 41 kB 00:00 2024-09-10T18:44:35.783 INFO:teuthology.orchestra.run.smithi162.stdout:(6/7): python3-routes-2.5.1-5.el9.noarch.rpm 3.7 MB/s | 188 kB 00:00 2024-09-10T18:44:35.892 INFO:teuthology.orchestra.run.smithi162.stdout:(7/7): ceph-mgr-dashboard-17.2.7-1606.g5be57395 8.2 MB/s | 3.4 MB 00:00 2024-09-10T18:44:35.896 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:35.896 INFO:teuthology.orchestra.run.smithi162.stdout:Total 8.2 MB/s | 3.7 MB 00:00 2024-09-10T18:44:35.896 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:35.904 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:35.904 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:35.913 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:44:35.983 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:35.984 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:36.281 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:36.352 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt-2.4.0-1.el9.noarch 1/7 2024-09-10T18:44:36.386 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jwt+crypto-2.4.0-1.el9.noarch 2/7 2024-09-10T18:44:36.440 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/7 2024-09-10T18:44:36.470 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:36.489 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/7 2024-09-10T18:44:36.518 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 5/7 2024-09-10T18:44:36.601 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:44:36.603 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout:======================================================================================= 2024-09-10T18:44:36.604 INFO:teuthology.orchestra.run.smithi107.stdout:Install 12 Packages 2024-09-10T18:44:36.605 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:36.605 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 38 M 2024-09-10T18:44:36.605 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 212 M 2024-09-10T18:44:36.605 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:36.986 INFO:teuthology.orchestra.run.smithi107.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 883 kB/s | 191 kB 00:00 2024-09-10T18:44:37.036 INFO:teuthology.orchestra.run.smithi107.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 655 kB/s | 33 kB 00:00 2024-09-10T18:44:37.070 INFO:teuthology.orchestra.run.smithi107.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-10T18:44:37.128 INFO:teuthology.orchestra.run.smithi107.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 313 kB/s | 18 kB 00:00 2024-09-10T18:44:37.229 INFO:teuthology.orchestra.run.smithi107.stdout:(5/12): ceph-mgr-diskprediction-local-17.2.7-16 16 MB/s | 7.4 MB 00:00 2024-09-10T18:44:37.254 INFO:teuthology.orchestra.run.smithi107.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 307 kB/s | 38 kB 00:00 2024-09-10T18:44:37.321 INFO:teuthology.orchestra.run.smithi107.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 10 MB/s | 3.0 MB 00:00 2024-09-10T18:44:37.346 INFO:teuthology.orchestra.run.smithi107.stdout:(8/12): python3-devel-3.9.19-8.el9.x86_64.rpm 2.6 MB/s | 245 kB 00:00 2024-09-10T18:44:37.405 INFO:teuthology.orchestra.run.smithi107.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-10T18:44:37.506 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 6/7 2024-09-10T18:44:37.564 INFO:teuthology.orchestra.run.smithi107.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 21 MB/s | 5.1 MB 00:00 2024-09-10T18:44:37.580 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:44:37.989 INFO:teuthology.orchestra.run.smithi107.stdout:(11/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 28 MB/s | 16 MB 00:00 2024-09-10T18:44:38.247 INFO:teuthology.orchestra.run.smithi107.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 4.9 MB/s | 4.9 MB 00:01 2024-09-10T18:44:38.248 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:38.248 INFO:teuthology.orchestra.run.smithi107.stdout:Total 23 MB/s | 38 MB 00:01 2024-09-10T18:44:38.599 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:38.624 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:38.625 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 7/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9. 1/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarc 2/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.n 3/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt+crypto-2.4.0-1.el9.noarch 4/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jwt-2.4.0-1.el9.noarch 5/7 2024-09-10T18:44:38.675 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 6/7 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 7/7 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-grafana-dashboards-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-dashboard-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-prometheus-alerts-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt-2.4.0-1.el9.noarch 2024-09-10T18:44:38.964 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jwt+crypto-2.4.0-1.el9.noarch 2024-09-10T18:44:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-10T18:44:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-10T18:44:38.965 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:38.965 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:39.151 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:39.151 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:39.236 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-diskprediction-local 2024-09-10T18:44:39.799 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:00 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:39.904 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:39.932 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 7.4 M 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-10T18:44:39.933 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout:======================================================================================= 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout:Install 12 Packages 2024-09-10T18:44:39.934 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:39.935 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 38 M 2024-09-10T18:44:39.935 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 212 M 2024-09-10T18:44:39.935 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:39.985 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:44:40.038 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:44:40.135 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:44:40.179 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:44:40.365 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:44:40.395 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:44:40.472 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:44:40.484 INFO:teuthology.orchestra.run.smithi162.stdout:(1/12): libquadmath-11.5.0-2.el9.x86_64.rpm 1.1 MB/s | 191 kB 00:00 2024-09-10T18:44:40.601 INFO:teuthology.orchestra.run.smithi162.stdout:(2/12): flexiblas-3.0.4-8.el9.x86_64.rpm 280 kB/s | 33 kB 00:00 2024-09-10T18:44:40.626 INFO:teuthology.orchestra.run.smithi162.stdout:(3/12): libgfortran-11.5.0-2.el9.x86_64.rpm 2.6 MB/s | 801 kB 00:00 2024-09-10T18:44:40.835 INFO:teuthology.orchestra.run.smithi162.stdout:(4/12): ceph-mgr-diskprediction-local-17.2.7-16 14 MB/s | 7.4 MB 00:00 2024-09-10T18:44:40.860 INFO:teuthology.orchestra.run.smithi162.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 78 kB/s | 18 kB 00:00 2024-09-10T18:44:40.952 INFO:teuthology.orchestra.run.smithi162.stdout:(6/12): openblas-0.3.26-2.el9.x86_64.rpm 329 kB/s | 38 kB 00:00 2024-09-10T18:44:41.103 INFO:teuthology.orchestra.run.smithi162.stdout:(7/12): python3-devel-3.9.19-8.el9.x86_64.rpm 1.6 MB/s | 245 kB 00:00 2024-09-10T18:44:41.111 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:44:41.203 INFO:teuthology.orchestra.run.smithi162.stdout:(8/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 5.0 MB/s | 3.0 MB 00:00 2024-09-10T18:44:41.241 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:44:41.261 INFO:teuthology.orchestra.run.smithi162.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 4.2 MB/s | 250 kB 00:00 2024-09-10T18:44:41.662 INFO:teuthology.orchestra.run.smithi162.stdout:(10/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 41 MB/s | 16 MB 00:00 2024-09-10T18:44:42.475 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:44:42.620 INFO:teuthology.orchestra.run.smithi162.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.4 MB/s | 5.1 MB 00:01 2024-09-10T18:44:42.745 INFO:teuthology.orchestra.run.smithi162.stdout:(12/12): openblas-openmp-0.3.26-2.el9.x86_64.rp 2.6 MB/s | 4.9 MB 00:01 2024-09-10T18:44:42.746 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:42.747 INFO:teuthology.orchestra.run.smithi162.stdout:Total 14 MB/s | 38 MB 00:02 2024-09-10T18:44:43.103 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:43.130 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:43.130 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:43.504 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:44:43.538 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:44:43.637 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:43.637 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:44.393 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:44.484 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-09-10T18:44:44.542 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:44:44.644 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:44:44.678 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 4/12 2024-09-10T18:44:44.867 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-0.3.26-2.el9.x86_64 5/12 2024-09-10T18:44:44.894 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 6/12 2024-09-10T18:44:44.977 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-09-10T18:44:45.640 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-09-10T18:44:45.759 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-09-10T18:44:46.152 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:44:46.152 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:44:46.152 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:44:46.152 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:44:46.153 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:44:46.546 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:46.547 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:46.947 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:44:47.004 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:44:47.515 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:11 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:47.647 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:44:47.649 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout:Install 13 Packages 2024-09-10T18:44:47.650 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:47.651 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.2 M 2024-09-10T18:44:47.651 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 26 M 2024-09-10T18:44:47.651 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:47.963 INFO:teuthology.orchestra.run.smithi107.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 211 kB/s | 26 kB 00:00 2024-09-10T18:44:47.988 INFO:teuthology.orchestra.run.smithi107.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 131 kB/s | 19 kB 00:00 2024-09-10T18:44:48.014 INFO:teuthology.orchestra.run.smithi107.stdout:(3/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 276 kB/s | 47 kB 00:00 2024-09-10T18:44:48.050 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-09-10T18:44:48.078 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:44:48.114 INFO:teuthology.orchestra.run.smithi107.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.7 MB/s | 222 kB 00:00 2024-09-10T18:44:48.139 INFO:teuthology.orchestra.run.smithi107.stdout:(5/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.0 MB/s | 159 kB 00:00 2024-09-10T18:44:48.181 INFO:teuthology.orchestra.run.smithi107.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 772 kB/s | 32 kB 00:00 2024-09-10T18:44:48.206 INFO:teuthology.orchestra.run.smithi107.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 799 kB/s | 54 kB 00:00 2024-09-10T18:44:48.231 INFO:teuthology.orchestra.run.smithi107.stdout:(8/13): python3-certifi-2023.05.07-4.el9.noarch 281 kB/s | 14 kB 00:00 2024-09-10T18:44:48.265 INFO:teuthology.orchestra.run.smithi107.stdout:(9/13): python3-pyasn1-modules-0.4.8-6.el9.noar 1.1 MB/s | 279 kB 00:00 2024-09-10T18:44:48.307 INFO:teuthology.orchestra.run.smithi107.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-10T18:44:48.332 INFO:teuthology.orchestra.run.smithi107.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 905 kB/s | 59 kB 00:00 2024-09-10T18:44:48.358 INFO:teuthology.orchestra.run.smithi107.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.8 MB/s | 90 kB 00:00 2024-09-10T18:44:48.391 INFO:teuthology.orchestra.run.smithi107.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 6.7 MB/s | 1.0 MB 00:00 2024-09-10T18:44:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:48.394 INFO:teuthology.orchestra.run.smithi107.stdout:Total 3.0 MB/s | 2.2 MB 00:00 2024-09-10T18:44:48.456 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:48.471 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:48.558 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:48.559 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:49.066 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:49.228 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:44:49.276 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:44:49.330 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:44:49.360 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:44:49.401 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:44:49.502 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:44:49.604 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:44:49.653 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:44:50.357 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:44:50.409 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:44:50.443 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:44:50.489 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:44:50.513 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:44:50.772 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 12/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57 1/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 2/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 3/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 7/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 8/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 9/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-09-10T18:44:50.773 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-diskprediction-local-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-10T18:44:51.190 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:51.191 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:51.288 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:44:51.289 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:44:51.601 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-rook 2024-09-10T18:44:51.660 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:44:51.660 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:51.660 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:44:51.661 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:51.662 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:51.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:44:52.181 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:13 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:52.317 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 47 k 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-10T18:44:52.318 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout:Install 13 Packages 2024-09-10T18:44:52.319 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:52.320 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.2 M 2024-09-10T18:44:52.320 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 26 M 2024-09-10T18:44:52.320 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:52.496 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:16 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:52.627 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:44:52.628 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout:Install 6 Packages 2024-09-10T18:44:52.629 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:52.630 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 7.0 M 2024-09-10T18:44:52.630 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 33 M 2024-09-10T18:44:52.630 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:52.696 INFO:teuthology.orchestra.run.smithi162.stdout:(1/13): ceph-mgr-rook-17.2.7-1606.g5be57395.el9 280 kB/s | 47 kB 00:00 2024-09-10T18:44:52.813 INFO:teuthology.orchestra.run.smithi162.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 89 kB/s | 26 kB 00:00 2024-09-10T18:44:52.838 INFO:teuthology.orchestra.run.smithi162.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 61 kB/s | 19 kB 00:00 2024-09-10T18:44:52.979 INFO:teuthology.orchestra.run.smithi107.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-09-10T18:44:53.020 INFO:teuthology.orchestra.run.smithi107.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 575 kB/s | 124 kB 00:00 2024-09-10T18:44:53.054 INFO:teuthology.orchestra.run.smithi107.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-10T18:44:53.088 INFO:teuthology.orchestra.run.smithi107.stdout:(4/6): python3-typing-extensions-4.12.2-2.el9.n 2.3 MB/s | 74 kB 00:00 2024-09-10T18:44:53.113 INFO:teuthology.orchestra.run.smithi107.stdout:(5/6): python3-natsort-7.1.1-5.el9.noarch.rpm 626 kB/s | 58 kB 00:00 2024-09-10T18:44:53.181 INFO:teuthology.orchestra.run.smithi162.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 432 kB/s | 159 kB 00:00 2024-09-10T18:44:53.196 INFO:teuthology.orchestra.run.smithi107.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-10T18:44:53.199 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:53.199 INFO:teuthology.orchestra.run.smithi107.stdout:Total 12 MB/s | 7.0 MB 00:00 2024-09-10T18:44:53.240 INFO:teuthology.orchestra.run.smithi162.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 409 kB/s | 222 kB 00:00 2024-09-10T18:44:53.268 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:53.279 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:53.279 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:53.281 INFO:teuthology.orchestra.run.smithi162.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 770 kB/s | 32 kB 00:00 2024-09-10T18:44:53.306 INFO:teuthology.orchestra.run.smithi162.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 596 kB/s | 279 kB 00:00 2024-09-10T18:44:53.332 INFO:teuthology.orchestra.run.smithi162.stdout:(8/13): python3-requests-oauthlib-1.3.0-12.el9. 355 kB/s | 54 kB 00:00 2024-09-10T18:44:53.366 INFO:teuthology.orchestra.run.smithi162.stdout:(9/13): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2024-09-10T18:44:53.378 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:53.379 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:53.407 INFO:teuthology.orchestra.run.smithi162.stdout:(10/13): python3-google-auth-2.34.0-1.el9.noarc 2.1 MB/s | 220 kB 00:00 2024-09-10T18:44:53.425 INFO:teuthology.orchestra.run.smithi162.stdout:(11/13): python3-websocket-client-1.2.3-2.el9.n 5.9 MB/s | 90 kB 00:00 2024-09-10T18:44:53.450 INFO:teuthology.orchestra.run.smithi162.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 722 kB/s | 59 kB 00:00 2024-09-10T18:44:53.483 INFO:teuthology.orchestra.run.smithi162.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 7.1 MB/s | 1.0 MB 00:00 2024-09-10T18:44:53.486 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:53.486 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.9 MB/s | 2.2 MB 00:01 2024-09-10T18:44:53.554 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:53.570 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:53.570 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:53.657 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:53.659 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:53.696 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:53.869 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:44:53.915 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:44:54.165 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:54.340 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-09-10T18:44:54.401 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-09-10T18:44:54.440 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:44:54.457 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-09-10T18:44:54.494 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-09-10T18:44:54.517 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:44:54.535 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/13 2024-09-10T18:44:54.573 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:44:54.602 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:44:54.638 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-09-10T18:44:54.740 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 7/13 2024-09-10T18:44:54.796 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-09-10T18:44:55.354 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:44:55.355 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:44:55.355 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:44:55.355 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:44:55.355 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:44:55.355 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:44:55.523 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-09-10T18:44:55.577 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-09-10T18:44:55.610 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:55.626 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:55.664 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-09-10T18:44:55.688 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:44:55.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-fuse 2024-09-10T18:44:56.476 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:20 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:56.534 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 13/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 1/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 9/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 10/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-09-10T18:44:56.535 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-09-10T18:44:56.612 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:56.612 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Installing dependencies: 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Install 2 Packages 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 893 k 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 2.6 M 2024-09-10T18:44:56.613 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:56.824 INFO:teuthology.orchestra.run.smithi107.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 711 kB/s | 80 kB 00:00 2024-09-10T18:44:56.879 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-rook-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-10T18:44:56.880 INFO:teuthology.orchestra.run.smithi162.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:56.881 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:44:56.991 INFO:teuthology.orchestra.run.smithi107.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-10T18:44:56.991 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:56.991 INFO:teuthology.orchestra.run.smithi107.stdout:Total 2.3 MB/s | 893 kB 00:00 2024-09-10T18:44:57.003 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:57.026 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:57.026 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:57.106 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:57.106 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:57.139 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-mgr-cephadm 2024-09-10T18:44:57.289 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:57.411 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:44:57.490 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:44:57.702 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:18 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:44:57.838 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 124 k 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-10T18:44:57.839 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout:Install 6 Packages 2024-09-10T18:44:57.840 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:44:57.841 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 7.0 M 2024-09-10T18:44:57.841 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 33 M 2024-09-10T18:44:57.841 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:44:58.148 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:44:58.148 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:44:58.245 INFO:teuthology.orchestra.run.smithi162.stdout:(1/6): python3-jinja2-2.11.3-6.el9.noarch.rpm 1.3 MB/s | 248 kB 00:00 2024-09-10T18:44:58.270 INFO:teuthology.orchestra.run.smithi162.stdout:(2/6): ceph-mgr-cephadm-17.2.7-1606.g5be57395.e 592 kB/s | 124 kB 00:00 2024-09-10T18:44:58.321 INFO:teuthology.orchestra.run.smithi162.stdout:(3/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 7.1 MB/s | 548 kB 00:00 2024-09-10T18:44:58.355 INFO:teuthology.orchestra.run.smithi162.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 689 kB/s | 58 kB 00:00 2024-09-10T18:44:58.380 INFO:teuthology.orchestra.run.smithi162.stdout:(5/6): python3-typing-extensions-4.12.2-2.el9.n 1.2 MB/s | 74 kB 00:00 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:58.420 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:44:58.722 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install ceph-volume 2024-09-10T18:44:59.313 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:23 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:44:59.422 INFO:teuthology.orchestra.run.smithi162.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.4 MB/s | 6.0 MB 00:01 2024-09-10T18:44:59.424 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:59.425 INFO:teuthology.orchestra.run.smithi162.stdout:Total 4.4 MB/s | 7.0 MB 00:01 2024-09-10T18:44:59.445 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repository Size 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:59.446 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 270 k 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 1.3 M 2024-09-10T18:44:59.447 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:44:59.496 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:44:59.506 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:44:59.506 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:44:59.605 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:44:59.606 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:44:59.671 INFO:teuthology.orchestra.run.smithi107.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:44:59.671 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:44:59.672 INFO:teuthology.orchestra.run.smithi107.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T18:44:59.672 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:44:59.680 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:44:59.680 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:44:59.710 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:44:59.711 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:44:59.910 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:44:59.922 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:44:59.990 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:00.006 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:00.006 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:45:00.006 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:45:00.006 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:00.070 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 1/6 2024-09-10T18:45:00.124 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-09-10T18:45:00.646 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-09-10T18:45:00.744 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-09-10T18:45:00.798 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 5/6 2024-09-10T18:45:00.828 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:45:00.968 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:00.968 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:00.969 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:00.969 INFO:teuthology.orchestra.run.smithi107.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:45:00.969 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:00.969 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:01.293 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados-devel 2024-09-10T18:45:01.667 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 6/6 2024-09-10T18:45:01.667 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 1/6 2024-09-10T18:45:01.667 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-09-10T18:45:01.667 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 3/6 2024-09-10T18:45:01.668 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-09-10T18:45:01.668 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-09-10T18:45:01.867 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:25 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 6/6 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-mgr-cephadm-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-10T18:45:01.942 INFO:teuthology.orchestra.run.smithi162.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-10T18:45:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-10T18:45:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-10T18:45:01.943 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:01.943 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:02.010 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:45:02.012 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:45:02.013 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 126 k 2024-09-10T18:45:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 448 k 2024-09-10T18:45:02.013 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:45:02.178 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-fuse 2024-09-10T18:45:02.227 INFO:teuthology.orchestra.run.smithi107.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 591 kB/s | 126 kB 00:00 2024-09-10T18:45:02.228 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:02.228 INFO:teuthology.orchestra.run.smithi107.stdout:Total 585 kB/s | 126 kB 00:00 2024-09-10T18:45:02.228 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:45:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:45:02.234 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:45:02.298 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:45:02.298 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:45:02.458 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:45:02.549 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:02.752 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:23 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:02.886 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:02.887 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:02.887 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:45:02.887 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:02.887 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:02.887 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 813 k 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Installing dependencies: 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Install 2 Packages 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 893 k 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 2.6 M 2024-09-10T18:45:02.888 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:02.955 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:03.069 INFO:teuthology.orchestra.run.smithi162.stdout:(1/2): fuse-2.9.9-16.el9.x86_64.rpm 613 kB/s | 80 kB 00:00 2024-09-10T18:45:03.205 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:03.205 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:03.206 INFO:teuthology.orchestra.run.smithi107.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:03.206 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:03.206 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:03.225 INFO:teuthology.orchestra.run.smithi162.stdout:(2/2): ceph-fuse-17.2.7-1606.g5be57395.el9.x86_ 2.8 MB/s | 813 kB 00:00 2024-09-10T18:45:03.225 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:03.226 INFO:teuthology.orchestra.run.smithi162.stdout:Total 2.6 MB/s | 893 kB 00:00 2024-09-10T18:45:03.237 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:03.258 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:03.259 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:03.340 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:03.341 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:03.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs2 2024-09-10T18:45:03.530 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:03.670 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : fuse-2.9.9-16.el9.x86_64 1/2 2024-09-10T18:45:03.733 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:45:04.075 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:04.166 INFO:teuthology.orchestra.run.smithi107.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:04.210 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:04.211 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:04.211 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:04.300 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install libcephfs-devel 2024-09-10T18:45:04.377 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2/2 2024-09-10T18:45:04.378 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/2 2024-09-10T18:45:04.683 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 2/2 2024-09-10T18:45:04.684 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:04.684 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:04.684 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 fuse-2.9.9-16.el9.x86_64 2024-09-10T18:45:04.684 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:04.684 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:04.867 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:28 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:04.941 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install ceph-volume 2024-09-10T18:45:05.001 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:05.001 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:05.001 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-10T18:45:05.001 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:05.001 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:45:05.002 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:05.003 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 25 k 2024-09-10T18:45:05.003 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 110 k 2024-09-10T18:45:05.003 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:45:05.148 INFO:teuthology.orchestra.run.smithi107.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 172 kB/s | 25 kB 00:00 2024-09-10T18:45:05.148 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:05.149 INFO:teuthology.orchestra.run.smithi107.stdout:Total 170 kB/s | 25 kB 00:00 2024-09-10T18:45:05.149 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:45:05.152 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:45:05.152 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:45:05.153 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:45:05.154 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:45:05.232 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:45:05.338 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:05.524 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:26 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:05.664 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:05.664 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repository Size 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume noarch 2:17.2.7-1606.g5be57395.el9 ceph-noarch 270 k 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 270 k 2024-09-10T18:45:05.665 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 1.3 M 2024-09-10T18:45:05.666 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:05.704 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:05.889 INFO:teuthology.orchestra.run.smithi162.stdout:ceph-volume-17.2.7-1606.g5be57395.el9.noarch.rp 1.2 MB/s | 270 kB 00:00 2024-09-10T18:45:05.890 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:05.890 INFO:teuthology.orchestra.run.smithi162.stdout:Total 1.2 MB/s | 270 kB 00:00 2024-09-10T18:45:05.890 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:05.898 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:05.898 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:05.929 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:05.929 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:05.962 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:05.963 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:05.963 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:05.963 INFO:teuthology.orchestra.run.smithi107.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:05.963 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:05.963 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:06.141 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:06.155 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librados2 2024-09-10T18:45:06.216 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:06.232 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:45:06.232 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-10T18:45:06.232 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:06.719 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:30 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:06.809 INFO:teuthology.orchestra.run.smithi107.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:06.851 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:06.852 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:06.852 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:06.927 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install librbd1 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 1/1 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: ceph-volume-2:17.2.7-1606.g5be57395.el9.noarch 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:07.293 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:07.487 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:07.576 INFO:teuthology.orchestra.run.smithi107.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:07.616 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados-devel 2024-09-10T18:45:07.620 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:07.621 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:07.621 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:07.697 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rados 2024-09-10T18:45:08.199 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:29 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:08.263 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:08.334 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 126 k 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:08.335 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:08.336 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 126 k 2024-09-10T18:45:08.336 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 448 k 2024-09-10T18:45:08.336 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:08.353 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:08.396 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:08.397 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:08.397 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:08.470 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rgw 2024-09-10T18:45:08.538 INFO:teuthology.orchestra.run.smithi162.stdout:librados-devel-17.2.7-1606.g5be57395.el9.x86_64 627 kB/s | 126 kB 00:00 2024-09-10T18:45:08.538 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:08.539 INFO:teuthology.orchestra.run.smithi162.stdout:Total 622 kB/s | 126 kB 00:00 2024-09-10T18:45:08.539 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:08.544 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:08.544 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:08.608 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:08.609 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:08.761 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:08.868 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:09.034 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:09.123 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:09.165 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:09.166 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:09.166 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:09.242 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-cephfs 2024-09-10T18:45:09.322 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout: librados-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:09.585 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:09.811 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:33 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:09.850 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs2 2024-09-10T18:45:09.902 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:09.946 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:09.947 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:09.947 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:10.020 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install python3-rbd 2024-09-10T18:45:10.431 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:31 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:10.524 INFO:teuthology.orchestra.run.smithi162.stdout:Package libcephfs2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:10.569 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:10.570 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:10.571 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:10.582 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:10.655 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install libcephfs-devel 2024-09-10T18:45:10.670 INFO:teuthology.orchestra.run.smithi107.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:10.713 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:10.714 INFO:teuthology.orchestra.run.smithi107.stdout:Nothing to do. 2024-09-10T18:45:10.714 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:10.783 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-fuse 2024-09-10T18:45:11.225 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:32 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:11.345 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:11.360 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel x86_64 2:17.2.7-1606.g5be57395.el9 ceph 25 k 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:11.361 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:11.362 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:11.362 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:11.362 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 25 k 2024-09-10T18:45:11.362 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 110 k 2024-09-10T18:45:11.362 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:11.478 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:11.479 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:45:11.480 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:11.480 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 82 k 2024-09-10T18:45:11.480 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 222 k 2024-09-10T18:45:11.480 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:45:11.526 INFO:teuthology.orchestra.run.smithi162.stdout:libcephfs-devel-17.2.7-1606.g5be57395.el9.x86_6 152 kB/s | 25 kB 00:00 2024-09-10T18:45:11.526 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:11.527 INFO:teuthology.orchestra.run.smithi162.stdout:Total 151 kB/s | 25 kB 00:00 2024-09-10T18:45:11.527 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:11.530 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:11.531 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:11.532 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:11.532 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:11.623 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:11.668 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 435 kB/s | 82 kB 00:00 2024-09-10T18:45:11.668 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:11.669 INFO:teuthology.orchestra.run.smithi107.stdout:Total 432 kB/s | 82 kB 00:00 2024-09-10T18:45:11.669 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:45:11.675 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:45:11.675 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:45:11.730 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:11.739 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:45:11.739 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:45:11.878 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:45:11.990 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:12.127 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:12.357 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: libcephfs-devel-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:12.372 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:12.570 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librados2 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:12.608 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:12.887 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-mirror 2024-09-10T18:45:13.140 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:13.231 INFO:teuthology.orchestra.run.smithi162.stdout:Package librados2-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:13.275 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:13.276 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:13.276 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:13.347 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install librbd1 2024-09-10T18:45:13.466 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:13.605 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout: Package Arch Version Repo Size 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 2.9 M 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 12 M 2024-09-10T18:45:13.606 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:45:13.910 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:34 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:13.938 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 8.9 MB/s | 2.9 MB 00:00 2024-09-10T18:45:13.999 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:13.999 INFO:teuthology.orchestra.run.smithi107.stdout:Total 8.8 MB/s | 2.9 MB 00:00 2024-09-10T18:45:13.999 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:45:13.999 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:45:13.999 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:45:14.000 INFO:teuthology.orchestra.run.smithi162.stdout:Package librbd1-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:14.029 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:45:14.029 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:45:14.044 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:14.045 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:14.045 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:14.117 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rados 2024-09-10T18:45:14.440 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:45:14.497 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:45:14.522 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:14.684 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:35 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:14.774 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rados-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:14.816 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:14.817 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:14.817 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:14.885 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rgw 2024-09-10T18:45:15.455 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:36 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:15.545 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rgw-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:15.581 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:15.588 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:15.589 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:15.589 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:15.659 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-cephfs 2024-09-10T18:45:15.855 DEBUG:teuthology.orchestra.run.smithi107:> sudo yum -y install rbd-nbd 2024-09-10T18:45:16.231 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:37 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:16.320 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-cephfs-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:16.364 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:16.365 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:16.365 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:16.434 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install python3-rbd 2024-09-10T18:45:16.434 INFO:teuthology.orchestra.run.smithi107.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:43:36 PM UTC. 2024-09-10T18:45:16.569 INFO:teuthology.orchestra.run.smithi107.stdout:Dependencies resolved. 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: Package Architecture Version Repository Size 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Installing: 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction Summary 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:================================================================================ 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Install 1 Package 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Total download size: 165 k 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Installed size: 477 k 2024-09-10T18:45:16.570 INFO:teuthology.orchestra.run.smithi107.stdout:Downloading Packages: 2024-09-10T18:45:16.776 INFO:teuthology.orchestra.run.smithi107.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 804 kB/s | 165 kB 00:00 2024-09-10T18:45:16.776 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:16.777 INFO:teuthology.orchestra.run.smithi107.stdout:Total 799 kB/s | 165 kB 00:00 2024-09-10T18:45:16.777 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction check 2024-09-10T18:45:16.784 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction check succeeded. 2024-09-10T18:45:16.784 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction test 2024-09-10T18:45:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:Transaction test succeeded. 2024-09-10T18:45:16.848 INFO:teuthology.orchestra.run.smithi107.stdout:Running transaction 2024-09-10T18:45:17.005 INFO:teuthology.orchestra.run.smithi107.stdout: Preparing : 1/1 2024-09-10T18:45:17.007 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:17.090 INFO:teuthology.orchestra.run.smithi107.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:17.097 INFO:teuthology.orchestra.run.smithi162.stdout:Package python3-rbd-2:17.2.7-1606.g5be57395.el9.x86_64 is already installed. 2024-09-10T18:45:17.139 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:17.140 INFO:teuthology.orchestra.run.smithi162.stdout:Nothing to do. 2024-09-10T18:45:17.141 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:17.240 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-fuse 2024-09-10T18:45:17.540 INFO:teuthology.orchestra.run.smithi107.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout:Installed: 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:45:17.795 INFO:teuthology.orchestra.run.smithi107.stdout:Complete! 2024-09-10T18:45:17.809 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:38 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:17.942 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:17.942 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse x86_64 2:17.2.7-1606.g5be57395.el9 ceph 82 k 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:17.943 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:17.944 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 82 k 2024-09-10T18:45:17.944 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 222 k 2024-09-10T18:45:17.944 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:18.044 DEBUG:teuthology.parallel:result is None 2024-09-10T18:45:18.146 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-fuse-17.2.7-1606.g5be57395.el9.x86_64.rpm 405 kB/s | 82 kB 00:00 2024-09-10T18:45:18.146 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:18.147 INFO:teuthology.orchestra.run.smithi162.stdout:Total 402 kB/s | 82 kB 00:00 2024-09-10T18:45:18.147 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:18.153 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:18.153 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:18.217 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:18.217 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:18.361 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:18.468 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:18.868 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-fuse-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:19.139 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:19.381 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-mirror 2024-09-10T18:45:19.966 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:40 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:20.109 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:20.109 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: Package Arch Version Repo Size 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror x86_64 2:17.2.7-1606.g5be57395.el9 ceph 2.9 M 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 2.9 M 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 12 M 2024-09-10T18:45:20.110 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:20.433 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-mirror-17.2.7-1606.g5be57395.el9.x86_64.rpm 9.1 MB/s | 2.9 MB 00:00 2024-09-10T18:45:20.434 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:20.434 INFO:teuthology.orchestra.run.smithi162.stdout:Total 9.1 MB/s | 2.9 MB 00:00 2024-09-10T18:45:20.434 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:20.443 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:20.443 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:20.524 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:20.524 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:20.938 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:20.992 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:21.017 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:21.017 INFO:teuthology.orchestra.run.smithi162.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-10T18:45:21.018 INFO:teuthology.orchestra.run.smithi162.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-10T18:45:21.018 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:45:21.018 INFO:teuthology.orchestra.run.smithi162.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-10T18:45:21.018 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:22.197 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:22.197 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:22.197 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:22.197 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-mirror-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:22.197 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:22.198 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:22.422 DEBUG:teuthology.orchestra.run.smithi162:> sudo yum -y install rbd-nbd 2024-09-10T18:45:23.001 INFO:teuthology.orchestra.run.smithi162.stdout:Last metadata expiration check: 0:01:43 ago on Tue 10 Sep 2024 06:43:39 PM UTC. 2024-09-10T18:45:23.138 INFO:teuthology.orchestra.run.smithi162.stdout:Dependencies resolved. 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout: Package Architecture Version Repository Size 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:Installing: 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd x86_64 2:17.2.7-1606.g5be57395.el9 ceph 165 k 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction Summary 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:================================================================================ 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout:Install 1 Package 2024-09-10T18:45:23.139 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:23.140 INFO:teuthology.orchestra.run.smithi162.stdout:Total download size: 165 k 2024-09-10T18:45:23.140 INFO:teuthology.orchestra.run.smithi162.stdout:Installed size: 477 k 2024-09-10T18:45:23.140 INFO:teuthology.orchestra.run.smithi162.stdout:Downloading Packages: 2024-09-10T18:45:23.355 INFO:teuthology.orchestra.run.smithi162.stdout:rbd-nbd-17.2.7-1606.g5be57395.el9.x86_64.rpm 769 kB/s | 165 kB 00:00 2024-09-10T18:45:23.356 INFO:teuthology.orchestra.run.smithi162.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:45:23.356 INFO:teuthology.orchestra.run.smithi162.stdout:Total 764 kB/s | 165 kB 00:00 2024-09-10T18:45:23.356 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction check 2024-09-10T18:45:23.364 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction check succeeded. 2024-09-10T18:45:23.364 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction test 2024-09-10T18:45:23.430 INFO:teuthology.orchestra.run.smithi162.stdout:Transaction test succeeded. 2024-09-10T18:45:23.430 INFO:teuthology.orchestra.run.smithi162.stdout:Running transaction 2024-09-10T18:45:23.587 INFO:teuthology.orchestra.run.smithi162.stdout: Preparing : 1/1 2024-09-10T18:45:23.694 INFO:teuthology.orchestra.run.smithi162.stdout: Installing : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:24.146 INFO:teuthology.orchestra.run.smithi162.stdout: Running scriptlet: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:24.413 INFO:teuthology.orchestra.run.smithi162.stdout: Verifying : rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 1/1 2024-09-10T18:45:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:24.414 INFO:teuthology.orchestra.run.smithi162.stdout:Installed: 2024-09-10T18:45:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: rbd-nbd-2:17.2.7-1606.g5be57395.el9.x86_64 2024-09-10T18:45:24.414 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:45:24.414 INFO:teuthology.orchestra.run.smithi162.stdout:Complete! 2024-09-10T18:45:24.661 DEBUG:teuthology.parallel:result is None 2024-09-10T18:45:24.661 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:45:24.803 DEBUG:teuthology.orchestra.run.smithi107:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:45:24.829 INFO:teuthology.orchestra.run.smithi107.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:45:24.829 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:45:24.829 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:45:24.832 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:45:24.975 DEBUG:teuthology.orchestra.run.smithi162:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-10T18:45:24.999 INFO:teuthology.orchestra.run.smithi162.stdout:17.2.7-1606.g5be57395.el9 2024-09-10T18:45:24.999 INFO:teuthology.packaging:The installed version of ceph is 17.2.7-1606.g5be57395.el9 2024-09-10T18:45:25.000 INFO:teuthology.task.install:The correct ceph version 17.2.7-1606.g5be57395 is installed. 2024-09-10T18:45:25.002 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-10T18:45:25.002 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:45:25.003 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:45:25.033 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:45:25.033 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-10T18:45:25.066 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-10T18:45:25.066 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:45:25.066 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:45:25.102 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:45:25.170 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:45:25.170 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/daemon-helper 2024-09-10T18:45:25.197 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-10T18:45:25.262 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-10T18:45:25.263 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:45:25.263 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:45:25.290 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:45:25.357 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:45:25.357 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-10T18:45:25.383 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-10T18:45:25.471 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-10T18:45:25.472 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:45:25.472 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:45:25.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:45:25.562 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:45:25.562 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/usr/bin/stdin-killer 2024-09-10T18:45:25.588 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-10T18:45:25.653 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-10T18:45:25.742 INFO:tasks.cephadm:Config: {'conf': {'mgr': {'debug mgr': 20, 'debug ms': 1}, 'client': {'debug ms': 1}, 'global': {'mon election default strategy': 3, 'ms type': 'async'}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon warn on pool no app': False}, 'osd': {'debug ms': 1, 'debug osd': 20, 'osd class default list': '*', 'osd class load list': '*', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'reached quota', 'but it is still running', 'overall HEALTH_', '\\(POOL_FULL\\)', '\\(SMALLER_PGP_NUM\\)', '\\(CACHE_POOL_NO_HIT_SET\\)', '\\(CACHE_POOL_NEAR_FULL\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)'], 'sha1': '5be57395ea235535910512ae73ba8e7ff25fe588', 'cephadm_mode': 'root'} 2024-09-10T18:45:25.742 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:45:25.743 INFO:tasks.cephadm:Cluster fsid is d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:45:25.743 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-10T18:45:25.743 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.107', 'mon.c': '[v2:172.21.15.107:3301,v1:172.21.15.107:6790]', 'mon.b': '172.21.15.162'} 2024-09-10T18:45:25.743 INFO:tasks.cephadm:First mon is mon.a on smithi107 2024-09-10T18:45:25.744 INFO:tasks.cephadm:First mgr is y 2024-09-10T18:45:25.744 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-10T18:45:25.744 DEBUG:teuthology.orchestra.run.smithi107:> sudo hostname $(hostname -s) 2024-09-10T18:45:25.769 DEBUG:teuthology.orchestra.run.smithi162:> sudo hostname $(hostname -s) 2024-09-10T18:45:25.795 INFO:tasks.cephadm:Downloading cephadm (repo https://git.ceph.com/ceph-ci.git ref 5be57395ea235535910512ae73ba8e7ff25fe588)... 2024-09-10T18:45:25.795 DEBUG:teuthology.orchestra.run.smithi107:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:45:25.824 INFO:teuthology.orchestra.run.smithi107.stderr:Cloning into 'testrepo'... 2024-09-10T18:46:25.140 INFO:teuthology.orchestra.run.smithi107.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:46 /home/ubuntu/cephtest/cephadm 2024-09-10T18:46:25.141 DEBUG:teuthology.orchestra.run.smithi162:> git clone https://git.ceph.com/ceph-ci.git testrepo && cd testrepo && git show 5be57395ea235535910512ae73ba8e7ff25fe588:src/cephadm/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-10T18:46:25.161 INFO:teuthology.orchestra.run.smithi162.stderr:Cloning into 'testrepo'... 2024-09-10T18:47:22.408 INFO:teuthology.orchestra.run.smithi162.stderr:Updating files: 96% (12280/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-10T18:47:22.441 INFO:teuthology.orchestra.run.smithi162.stdout:-rw-r--r--. 1 ubuntu ubuntu 378585 Sep 10 18:47 /home/ubuntu/cephtest/cephadm 2024-09-10T18:47:22.442 DEBUG:teuthology.orchestra.run.smithi107:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:47:22.463 DEBUG:teuthology.orchestra.run.smithi162:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-10T18:47:22.509 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 on all hosts... 2024-09-10T18:47:22.510 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:47:22.512 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 pull 2024-09-10T18:47:24.068 INFO:teuthology.orchestra.run.smithi107.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:47:26.127 INFO:teuthology.orchestra.run.smithi162.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:48:05.846 INFO:teuthology.orchestra.run.smithi107.stdout:{ 2024-09-10T18:48:05.846 INFO:teuthology.orchestra.run.smithi107.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:48:05.847 INFO:teuthology.orchestra.run.smithi107.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:48:05.847 INFO:teuthology.orchestra.run.smithi107.stdout: "repo_digests": [ 2024-09-10T18:48:05.847 INFO:teuthology.orchestra.run.smithi107.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:48:05.847 INFO:teuthology.orchestra.run.smithi107.stdout: ] 2024-09-10T18:48:05.847 INFO:teuthology.orchestra.run.smithi107.stdout:} 2024-09-10T18:48:08.249 INFO:teuthology.orchestra.run.smithi162.stdout:{ 2024-09-10T18:48:08.249 INFO:teuthology.orchestra.run.smithi162.stdout: "ceph_version": "ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable)", 2024-09-10T18:48:08.249 INFO:teuthology.orchestra.run.smithi162.stdout: "image_id": "4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9", 2024-09-10T18:48:08.249 INFO:teuthology.orchestra.run.smithi162.stdout: "repo_digests": [ 2024-09-10T18:48:08.250 INFO:teuthology.orchestra.run.smithi162.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16" 2024-09-10T18:48:08.250 INFO:teuthology.orchestra.run.smithi162.stdout: ] 2024-09-10T18:48:08.250 INFO:teuthology.orchestra.run.smithi162.stdout:} 2024-09-10T18:48:08.270 DEBUG:teuthology.orchestra.run.smithi107:> sudo mkdir -p /etc/ceph 2024-09-10T18:48:08.303 DEBUG:teuthology.orchestra.run.smithi162:> sudo mkdir -p /etc/ceph 2024-09-10T18:48:08.332 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 777 /etc/ceph 2024-09-10T18:48:08.373 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 777 /etc/ceph 2024-09-10T18:48:08.399 INFO:tasks.cephadm:Writing seed config... 2024-09-10T18:48:08.400 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-10T18:48:08.400 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [global] mon election default strategy = 3 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [global] ms type = async 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [mon] mon warn on pool no app = False 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [osd] osd class default list = * 2024-09-10T18:48:08.401 INFO:tasks.cephadm: override: [osd] osd class load list = * 2024-09-10T18:48:08.402 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-10T18:48:08.402 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:48:08.402 DEBUG:teuthology.orchestra.run.smithi107:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-10T18:48:08.430 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = d7c9fa7c-6fa4-11ef-bcea-c7b262605968 mon election default strategy = 3 ms type = async [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true debug ms = 1 debug osd = 20 osd class default list = * osd class load list = * [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon warn on pool no app = False [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] debug ms = 1 2024-09-10T18:48:08.430 DEBUG:teuthology.orchestra.run.smithi107:mon.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service 2024-09-10T18:48:08.473 DEBUG:teuthology.orchestra.run.smithi107:mgr.y> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y.service 2024-09-10T18:48:08.515 INFO:tasks.cephadm:Bootstrapping... 2024-09-10T18:48:08.515 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 -v bootstrap --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id y --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.107 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:48:08.716 INFO:teuthology.orchestra.run.smithi107.stdout:-------------------------------------------------------------------------------- 2024-09-10T18:48:08.717 INFO:teuthology.orchestra.run.smithi107.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588', '-v', 'bootstrap', '--fsid', 'd7c9fa7c-6fa4-11ef-bcea-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'y', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.107', '--skip-admin-label'] 2024-09-10T18:48:08.739 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:48:08.740 INFO:teuthology.orchestra.run.smithi107.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-10T18:48:08.740 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying podman|docker is present... 2024-09-10T18:48:08.767 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:48:08.768 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying lvm2 is present... 2024-09-10T18:48:08.768 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying time synchronization is in place... 2024-09-10T18:48:08.775 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:48:08.776 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:48:08.782 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:48:08.782 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:48:08.789 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-10T18:48:08.795 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-10T18:48:08.795 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:48:08.795 INFO:teuthology.orchestra.run.smithi107.stdout:Repeating the final host check... 2024-09-10T18:48:08.817 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 5.2.2 2024-09-10T18:48:08.817 INFO:teuthology.orchestra.run.smithi107.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-10T18:48:08.817 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl is present 2024-09-10T18:48:08.818 INFO:teuthology.orchestra.run.smithi107.stdout:lvcreate is present 2024-09-10T18:48:08.824 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-10T18:48:08.824 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-10T18:48:08.831 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-10T18:48:08.831 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:48:08.837 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout enabled 2024-09-10T18:48:08.844 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout active 2024-09-10T18:48:08.844 INFO:teuthology.orchestra.run.smithi107.stdout:Unit chronyd.service is enabled and running 2024-09-10T18:48:08.844 INFO:teuthology.orchestra.run.smithi107.stdout:Host looks OK 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Cluster fsid: d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Acquiring lock 140524014941088 on /run/cephadm/d7c9fa7c-6fa4-11ef-bcea-c7b262605968.lock 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Lock 140524014941088 acquired on /run/cephadm/d7c9fa7c-6fa4-11ef-bcea-c7b262605968.lock 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 3300 ... 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying IP 172.21.15.107 port 6789 ... 2024-09-10T18:48:08.845 INFO:teuthology.orchestra.run.smithi107.stdout:Base mon IP(s) is [172.21.15.107:3300, 172.21.15.107:6789], mon addrv is [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-10T18:48:08.849 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.107 metric 100 2024-09-10T18:48:08.849 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.107 metric 100 2024-09-10T18:48:08.852 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-10T18:48:08.852 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-09-10T18:48:08.852 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:fe8f:47eb/64 scope link noprefixroute 2024-09-10T18:48:08.855 INFO:teuthology.orchestra.run.smithi107.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-10T18:48:08.856 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-10T18:48:08.856 INFO:teuthology.orchestra.run.smithi107.stdout:Mon IP `172.21.15.107` is in CIDR network `172.21.0.0/20` 2024-09-10T18:48:08.856 INFO:teuthology.orchestra.run.smithi107.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-10T18:48:08.856 INFO:teuthology.orchestra.run.smithi107.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-10T18:48:08.857 INFO:teuthology.orchestra.run.smithi107.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:48:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stdout 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:48:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588... 2024-09-10T18:48:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Getting image source signatures 2024-09-10T18:48:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:5df13064c03763eb21a89b455ab82ba4ec435f51a3f340cae76514aaf2996d4c 2024-09-10T18:48:09.244 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-10T18:48:09.245 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Copying config sha256:4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 2024-09-10T18:48:09.245 INFO:teuthology.orchestra.run.smithi107.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-10T18:48:10.224 INFO:teuthology.orchestra.run.smithi107.stdout:ceph: stdout ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:48:10.224 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph version: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable) 2024-09-10T18:48:10.224 INFO:teuthology.orchestra.run.smithi107.stdout:Extracting ceph user uid/gid from container image... 2024-09-10T18:48:11.180 INFO:teuthology.orchestra.run.smithi107.stdout:stat: stdout 167 167 2024-09-10T18:48:11.180 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial keys... 2024-09-10T18:48:12.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBrlOBm+TZ5HhAAUkKve0slvSyjVQweEZv0SQ== 2024-09-10T18:48:13.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBslOBms4rAIhAA9LJTy/bpj+ItJTEW3POdWw== 2024-09-10T18:48:14.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph-authtool: stdout AQBtlOBmEx0GIhAA4Rz3GdIYRGyYPPxti3IkKQ== 2024-09-10T18:48:14.262 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial monmap... 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = octopus 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:monmaptool for a [v2:172.21.15.107:3300,v1:172.21.15.107:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:setting min_mon_release = octopus 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: set fsid to d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:48:15.243 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mon... 2024-09-10T18:48:16.131 INFO:teuthology.orchestra.run.smithi107.stdout:create mon.a on 2024-09-10T18:48:16.330 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-10T18:48:16.468 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-10T18:48:16.613 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target. 2024-09-10T18:48:16.613 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target → /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target. 2024-09-10T18:48:16.779 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a 2024-09-10T18:48:16.779 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service: Unit ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service not loaded. 2024-09-10T18:48:16.940 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target.wants/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service → /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@.service. 2024-09-10T18:48:17.435 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:17 smithi107 podman[28896]: 2024-09-10 18:48:17.396455146 +0000 UTC m=+0.339577544 container start d8c14ccafdf0112500bb166cc45789ddae0d4709197976899458b9522d1b16b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-10T18:48:17.435 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:17 smithi107 ceph-mon[28933]: mkfs d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:17.511 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:48:17.512 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon to start... 2024-09-10T18:48:17.519 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mon... 2024-09-10T18:48:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:17 smithi107 ceph-mon[28933]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:48:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:17 smithi107 bash[28896]: d8c14ccafdf0112500bb166cc45789ddae0d4709197976899458b9522d1b16b6 2024-09-10T18:48:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:17 smithi107 systemd[1]: Started Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: fsmap 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: mgrmap e1: no daemons active 2024-09-10T18:48:18.546 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:18 smithi107 ceph-mon[28933]: from='client.? 172.21.15.107:0/1830371196' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-09-10T18:48:18.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout cluster: 2024-09-10T18:48:18.553 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout id: d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout services: 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.57138s) 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout data: 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-10T18:48:18.554 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout pgs: 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.932+0000 7ff2964d3640 1 Processor -- start 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.932+0000 7ff2964d3640 1 -- start start 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.932+0000 7ff2964d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.932+0000 7ff2964d3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff290100ca0 con 0x7ff290100350 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.933+0000 7ff28ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.933+0000 7ff28ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:44998/0 (socket says 172.21.15.107:44998) 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.933+0000 7ff28ffff640 1 -- 172.21.15.107:0/108142167 learned_addr learned my addr 172.21.15.107:0/108142167 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.933+0000 7ff28ffff640 1 -- 172.21.15.107:0/108142167 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff290100de0 con 0x7ff290100350 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.935+0000 7ff28ffff640 1 --2- 172.21.15.107:0/108142167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff280009840 tx=0x7ff2800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be2933128d0e4cbe server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:18.555 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.935+0000 7ff28effd640 1 -- 172.21.15.107:0/108142167 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff28002add0 con 0x7ff290100350 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.935+0000 7ff28effd640 1 -- 172.21.15.107:0/108142167 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280032440 con 0x7ff290100350 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.935+0000 7ff28effd640 1 -- 172.21.15.107:0/108142167 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff280032750 con 0x7ff290100350 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- 172.21.15.107:0/108142167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 msgr2=0x7ff290100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 --2- 172.21.15.107:0/108142167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7ff280009840 tx=0x7ff2800091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- 172.21.15.107:0/108142167 shutdown_connections 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 --2- 172.21.15.107:0/108142167 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff290100760 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- 172.21.15.107:0/108142167 >> 172.21.15.107:0/108142167 conn(0x7ff2900fbb50 msgr2=0x7ff2900fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- 172.21.15.107:0/108142167 shutdown_connections 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- 172.21.15.107:0/108142167 wait complete. 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 Processor -- start 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.936+0000 7ff2964d3640 1 -- start start 2024-09-10T18:48:18.556 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff2964d3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff2964d3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff29010ed90 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45004/0 (socket says 172.21.15.107:45004) 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28ffff640 1 -- 172.21.15.107:0/1830371196 learned_addr learned my addr 172.21.15.107:0/1830371196 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28ffff640 1 -- 172.21.15.107:0/1830371196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff280009450 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28ffff640 1 --2- 172.21.15.107:0/1830371196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff28002a700 tx=0x7ff280032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff280031a80 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff280031be0 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff29010ef90 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff28003b400 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff29010f3a0 con 0x7ff290100350 2024-09-10T18:48:18.557 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7ff280039040 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.937+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7ff280041030 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.938+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff29004b720 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.940+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7ff280037070 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.998+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7ff29010fb50 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.998+0000 7ff28cff9640 1 -- 172.21.15.107:0/1830371196 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+316 (secure 0 0 0) 0x7ff28003b980 con 0x7ff290100350 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 msgr2=0x7ff29010e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 --2- 172.21.15.107:0/1830371196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff28002a700 tx=0x7ff280032000 comp rx=0 tx=0).stop 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 shutdown_connections 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 --2- 172.21.15.107:0/1830371196 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff290100350 0x7ff29010e850 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 >> 172.21.15.107:0/1830371196 conn(0x7ff2900fbb50 msgr2=0x7ff2901050d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 shutdown_connections 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:17.999+0000 7ff2964d3640 1 -- 172.21.15.107:0/1830371196 wait complete. 2024-09-10T18:48:18.558 INFO:teuthology.orchestra.run.smithi107.stdout:mon is available 2024-09-10T18:48:18.559 INFO:teuthology.orchestra.run.smithi107.stdout:Assimilating anything we can from ceph.conf... 2024-09-10T18:48:19.520 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:19.520 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:48:19.520 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:19.520 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:48:19.520 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.107:3300,v1:172.21.15.107:6789] 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f5118e6a640 1 Processor -- start 2024-09-10T18:48:19.521 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f5118e6a640 1 -- start start 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f5118e6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f5118e6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5114100d20 con 0x7f5114073c20 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f511259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f511259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45014/0 (socket says 172.21.15.107:45014) 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f511259c640 1 -- 172.21.15.107:0/572611399 learned_addr learned my addr 172.21.15.107:0/572611399 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:19.522 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.980+0000 7f511259c640 1 -- 172.21.15.107:0/572611399 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5114105870 con 0x7f5114073c20 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f511259c640 1 --2- 172.21.15.107:0/572611399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5104009840 tx=0x7f51040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8f130eaa1c3d3f3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f511159a640 1 -- 172.21.15.107:0/572611399 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510402add0 con 0x7f5114073c20 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f511159a640 1 -- 172.21.15.107:0/572611399 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104032440 con 0x7f5114073c20 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f511159a640 1 -- 172.21.15.107:0/572611399 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5104032750 con 0x7f5114073c20 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 -- 172.21.15.107:0/572611399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 msgr2=0x7f5114105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 --2- 172.21.15.107:0/572611399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f5104009840 tx=0x7f51040091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 -- 172.21.15.107:0/572611399 shutdown_connections 2024-09-10T18:48:19.523 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 --2- 172.21.15.107:0/572611399 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114105330 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 -- 172.21.15.107:0/572611399 >> 172.21.15.107:0/572611399 conn(0x7f51140faa00 msgr2=0x7f51140fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.981+0000 7f5118e6a640 1 -- 172.21.15.107:0/572611399 shutdown_connections 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f5118e6a640 1 -- 172.21.15.107:0/572611399 wait complete. 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f5118e6a640 1 Processor -- start 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f5118e6a640 1 -- start start 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f5118e6a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f5118e6a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5114110f50 con 0x7f5114073c20 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f511259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f511259c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45022/0 (socket says 172.21.15.107:45022) 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.982+0000 7f511259c640 1 -- 172.21.15.107:0/3985551665 learned_addr learned my addr 172.21.15.107:0/3985551665 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f511259c640 1 -- 172.21.15.107:0/3985551665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5104009450 con 0x7f5114073c20 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f511259c640 1 --2- 172.21.15.107:0/3985551665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f510402a700 tx=0x7f5104032000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:19.524 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5104031a80 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5104031be0 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f510403b400 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5114111150 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5114111560 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7f5104039040 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.983+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f5104041030 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.984+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f511404b720 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:18.986+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7f5104037070 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.041+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f5114105a60 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.044+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 7 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f5104046820 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.044+0000 7f510affd640 1 -- 172.21.15.107:0/3985551665 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) v1 ==== 70+0+376 (secure 0 0 0) 0x7f5104055190 con 0x7f5114073c20 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 msgr2=0x7f5114110a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:19.525 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 --2- 172.21.15.107:0/3985551665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7f510402a700 tx=0x7f5104032000 comp rx=0 tx=0).stop 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 shutdown_connections 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 --2- 172.21.15.107:0/3985551665 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5114073c20 0x7f5114110a10 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 >> 172.21.15.107:0/3985551665 conn(0x7f51140faa00 msgr2=0x7f51140fb7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 shutdown_connections 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.045+0000 7f5118e6a640 1 -- 172.21.15.107:0/3985551665 wait complete. 2024-09-10T18:48:19.526 INFO:teuthology.orchestra.run.smithi107.stdout:Generating new minimal ceph.conf... 2024-09-10T18:48:19.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:19 smithi107 ceph-mon[28933]: from='client.? 172.21.15.107:0/3985551665' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:48:19.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:19 smithi107 ceph-mon[28933]: from='client.? 172.21.15.107:0/3985551665' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-10T18:48:20.538 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.959+0000 7fc3f64e2640 1 Processor -- start 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.959+0000 7fc3f64e2640 1 -- start start 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3f64e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3f64e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f00ffb30 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45036/0 (socket says 172.21.15.107:45036) 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3effff640 1 -- 172.21.15.107:0/1680869403 learned_addr learned my addr 172.21.15.107:0/1680869403 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.960+0000 7fc3effff640 1 -- 172.21.15.107:0/1680869403 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3f00ffc70 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3effff640 1 --2- 172.21.15.107:0/1680869403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009840 tx=0x7fc3e00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ad31392241b69f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3eeffd640 1 -- 172.21.15.107:0/1680869403 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3e002add0 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3eeffd640 1 -- 172.21.15.107:0/1680869403 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fc3e0032860 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3eeffd640 1 -- 172.21.15.107:0/1680869403 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3e0031630 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1680869403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 msgr2=0x7fc3f00ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:20.539 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.961+0000 7fc3f64e2640 1 --2- 172.21.15.107:0/1680869403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0009840 tx=0x7fc3e00091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1680869403 shutdown_connections 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 --2- 172.21.15.107:0/1680869403 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f00ff5f0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1680869403 >> 172.21.15.107:0/1680869403 conn(0x7fc3f00faa00 msgr2=0x7fc3f00fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1680869403 shutdown_connections 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1680869403 wait complete. 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 Processor -- start 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- start start 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.962+0000 7fc3f64e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3f010e360 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3effff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45042/0 (socket says 172.21.15.107:45042) 2024-09-10T18:48:20.540 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3effff640 1 -- 172.21.15.107:0/1143214843 learned_addr learned my addr 172.21.15.107:0/1143214843 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3effff640 1 -- 172.21.15.107:0/1143214843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3e0009450 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3effff640 1 --2- 172.21.15.107:0/1143214843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0031ad0 tx=0x7fc3e002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3e003b440 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fc3e003b9c0 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3f010e560 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc3e003a3a0 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3f010efb0 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fc3e0039070 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.963+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fc3e0044450 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.964+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3f005ed70 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:19.966+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fc3e0049060 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.022+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fc3f0101060 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.541 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3ecff9640 1 -- 172.21.15.107:0/1143214843 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) v1 ==== 76+0+177 (secure 0 0 0) 0x7fc3e003f070 con 0x7fc3f00ff1e0 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 msgr2=0x7fc3f010de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 --2- 172.21.15.107:0/1143214843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc3e0031ad0 tx=0x7fc3e002c040 comp rx=0 tx=0).stop 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 shutdown_connections 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 --2- 172.21.15.107:0/1143214843 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc3f00ff1e0 0x7fc3f010de20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 >> 172.21.15.107:0/1143214843 conn(0x7fc3f00faa00 msgr2=0x7fc3f00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 shutdown_connections 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:20.023+0000 7fc3f64e2640 1 -- 172.21.15.107:0/1143214843 wait complete. 2024-09-10T18:48:20.542 INFO:teuthology.orchestra.run.smithi107.stdout:Restarting the monitor... 2024-09-10T18:48:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:20 smithi107 ceph-mon[28933]: from='client.? 172.21.15.107:0/1143214843' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:48:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:20 smithi107 systemd[1]: Stopping Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:48:21.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a[28910]: 2024-09-10T18:48:20.802+0000 7f8af6af6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T18:48:21.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a[28910]: 2024-09-10T18:48:20.802+0000 7f8af6af6640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-10T18:48:21.110 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:20 smithi107 podman[29252]: 2024-09-10 18:48:20.864649273 +0000 UTC m=+0.172680947 container died d8c14ccafdf0112500bb166cc45789ddae0d4709197976899458b9522d1b16b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T18:48:21.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 podman[29252]: 2024-09-10 18:48:21.208451008 +0000 UTC m=+0.516482680 container cleanup d8c14ccafdf0112500bb166cc45789ddae0d4709197976899458b9522d1b16b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:48:21.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 bash[29252]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a 2024-09-10T18:48:21.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 podman[29271]: 2024-09-10 18:48:21.38432658 +0000 UTC m=+0.512941019 container remove d8c14ccafdf0112500bb166cc45789ddae0d4709197976899458b9522d1b16b6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:48:21.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service: Deactivated successfully. 2024-09-10T18:48:21.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 systemd[1]: Stopped Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:48:21.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 systemd[1]: Starting Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:48:21.950 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 podman[29367]: 2024-09-10 18:48:21.835336037 +0000 UTC m=+0.102027617 container create d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-09-10T18:48:21.951 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:21 smithi107 podman[29367]: 2024-09-10 18:48:21.749026521 +0000 UTC m=+0.015718109 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 2024-09-10T18:48:22.151 INFO:teuthology.orchestra.run.smithi107.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-10T18:48:22.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 podman[29367]: 2024-09-10 18:48:22.06252125 +0000 UTC m=+0.329212838 container init d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True) 2024-09-10T18:48:22.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 podman[29367]: 2024-09-10 18:48:22.066888849 +0000 UTC m=+0.333580430 container start d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-10T18:48:22.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:48:22.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:48:22.369 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: pidfile_write: ignore empty --pid-file 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: load: jerasure load: lrc 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: DB SUMMARY 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: DB Session ID: VFKAE1QEMH7X2IGYY6V9 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: MANIFEST file: MANIFEST-000009 size: 131 Bytes 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000010.log size: 85078 ; 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:48:22.370 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.env: 0x55f8182f1440 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.fs: Posix File System 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.info_log: 0x55f818f28ec0 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.statistics: (nil) 2024-09-10T18:48:22.371 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.use_fsync: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:48:22.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.db_log_dir: 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-a/store.db 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:48:22.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.write_buffer_manager: 0x55f818fe6780 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.unordered_write: 0 2024-09-10T18:48:22.374 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.row_cache: None 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.wal_filter: None 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:48:22.375 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:48:22.376 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_open_files: -1 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:48:22.377 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Compression algorithms supported: 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kZSTD supported: 0 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:48:22.378 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.merge_operator: 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_filter: None 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55f818ecfa38) 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_type: 0 2024-09-10T18:48:22.379 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_index_type: 0 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_shortening: 1 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: hash_index_allow_collision: 1 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: checksum: 1 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: no_block_cache: 0 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache: 0x55f818f051f0 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_options: 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: capacity : 536870912 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: num_shard_bits : 4 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: strict_capacity_limit : 0 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:48:22.380 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_cache_compressed: (nil) 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: persistent_cache: (nil) 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size: 4096 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_size_deviation: 10 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_restart_interval: 16 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: index_block_restart_interval: 1 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: metadata_block_size: 4096 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: partition_filters: 0 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: use_delta_encoding: 1 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: whole_key_filtering: 1 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: verify_compression: 0 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: format_version: 4 2024-09-10T18:48:22.381 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_index_compression: 1 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout: block_align: 0 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression: NoCompression 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.num_levels: 7 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:48:22.382 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:48:22.383 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:48:22.384 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:48:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:48:22.386 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.ttl: 2592000 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:48:22.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000009 succeeded,manifest_file_number is 9, next_file_number is 11, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 5 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/version_set.cc:4082] Creating manifest 13 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994102090411, "job": 1, "event": "recovery_started", "wal_files": [10]} 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #10 mode 2 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994102091482, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 14, "file_size": 81522, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 79761, "index_size": 223, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 581, "raw_key_size": 10324, "raw_average_key_size": 47, "raw_value_size": 73932, "raw_average_value_size": 340, "num_data_blocks": 10, "num_entries": 217, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725994102, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "f9b5ed4b-6550-4f63-bfee-05b488e373ec", "db_session_id": "VFKAE1QEMH7X2IGYY6V9"}} 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/version_set.cc:4082] Creating manifest 15 2024-09-10T18:48:22.388 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994102091921, "job": 1, "event": "recovery_finished"} 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000010.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x55f818f38700 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: DB pointer 0x55f818f08000 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** DB Stats ** 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:48:22.389 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 81.40 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 81.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:48:22.390 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 18.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 18.56 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:48:22.391 INFO:journalctl@ceph.mon.a.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: L0 2/0 81.40 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: Sum 2/0 81.40 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 109.2 0.00 0.00 1 0.001 0 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:48:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: Cumulative compaction: 0.00 GB write, 18.42 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: starting mon.a rank 0 at public addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] at bind addrs [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???) e1 preinit fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).mds e1 new map 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).mds e1 print_map 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: e1 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:48:22.393 INFO:journalctl@ceph.mon.a.smithi107.stdout: legacy client fscid: -1 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout: No filesystems configured 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 bash[29367]: d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 systemd[1]: Started Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: monmap e1: 1 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: fsmap 2024-09-10T18:48:22.394 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: osdmap e1: 0 total, 0 up, 0 in 2024-09-10T18:48:22.395 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:22 smithi107 ceph-mon[29404]: mgrmap e1: no daemons active 2024-09-10T18:48:23.160 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.601+0000 7fd159747640 1 Processor -- start 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.601+0000 7fd159747640 1 -- start start 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.601+0000 7fd159747640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.601+0000 7fd159747640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1540ffbc0 con 0x7fd1540ff270 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.602+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.602+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45050/0 (socket says 172.21.15.107:45050) 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.602+0000 7fd152ffd640 1 -- 172.21.15.107:0/398956087 learned_addr learned my addr 172.21.15.107:0/398956087 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.602+0000 7fd152ffd640 1 -- 172.21.15.107:0/398956087 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1540ffd00 con 0x7fd1540ff270 2024-09-10T18:48:23.161 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.604+0000 7fd152ffd640 1 --2- 172.21.15.107:0/398956087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00baf0 tx=0x7fd14c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94a8c08d5352756f server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.604+0000 7fd151ffb640 1 -- 172.21.15.107:0/398956087 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14c032640 con 0x7fd1540ff270 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.604+0000 7fd151ffb640 1 -- 172.21.15.107:0/398956087 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fd14c032bc0 con 0x7fd1540ff270 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.604+0000 7fd151ffb640 1 -- 172.21.15.107:0/398956087 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14c031a00 con 0x7fd1540ff270 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 -- 172.21.15.107:0/398956087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 msgr2=0x7fd1540ff680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 --2- 172.21.15.107:0/398956087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00baf0 tx=0x7fd14c00b460 comp rx=0 tx=0).stop 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 -- 172.21.15.107:0/398956087 shutdown_connections 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 --2- 172.21.15.107:0/398956087 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd1540ff680 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 -- 172.21.15.107:0/398956087 >> 172.21.15.107:0/398956087 conn(0x7fd1540faa70 msgr2=0x7fd1540fce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 -- 172.21.15.107:0/398956087 shutdown_connections 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.605+0000 7fd159747640 1 -- 172.21.15.107:0/398956087 wait complete. 2024-09-10T18:48:23.162 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd159747640 1 Processor -- start 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd159747640 1 -- start start 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd159747640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd159747640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd15410d160 con 0x7fd1540ff270 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd152ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:45066/0 (socket says 172.21.15.107:45066) 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd152ffd640 1 -- 172.21.15.107:0/545943107 learned_addr learned my addr 172.21.15.107:0/545943107 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd152ffd640 1 -- 172.21.15.107:0/545943107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd14c00b700 con 0x7fd1540ff270 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd152ffd640 1 --2- 172.21.15.107:0/545943107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00bac0 tx=0x7fd14c03d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14c032920 con 0x7fd1540ff270 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.606+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fd14c03b840 con 0x7fd1540ff270 2024-09-10T18:48:23.163 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd15410d360 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fd14c045590 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd15410d7d0 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fd14c045980 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fd14c04eaf0 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.607+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd15404b720 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.610+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fd15410d7d0 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.664+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) v1 -- 0x7fd154100ee0 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd13b7fe640 1 -- 172.21.15.107:0/545943107 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) v1 ==== 124+0+0 (secure 0 0 0) 0x7fd14c036020 con 0x7fd1540ff270 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 msgr2=0x7fd15410cc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd159747640 1 --2- 172.21.15.107:0/545943107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fd14c00bac0 tx=0x7fd14c03d040 comp rx=0 tx=0).stop 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 shutdown_connections 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd159747640 1 --2- 172.21.15.107:0/545943107 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd1540ff270 0x7fd15410cc20 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:23.164 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.668+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 >> 172.21.15.107:0/545943107 conn(0x7fd1540faa70 msgr2=0x7fd154103a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.669+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 shutdown_connections 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:22.669+0000 7fd159747640 1 -- 172.21.15.107:0/545943107 wait complete. 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:Creating mgr... 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-10T18:48:23.165 INFO:teuthology.orchestra.run.smithi107.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-10T18:48:23.343 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y 2024-09-10T18:48:23.343 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Failed to reset failed state of unit ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y.service: Unit ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y.service not loaded. 2024-09-10T18:48:23.493 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968.target.wants/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y.service → /etc/systemd/system/ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@.service. 2024-09-10T18:48:23.840 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/545943107' entity='client.admin' 2024-09-10T18:48:23.840 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:23 smithi107 podman[29641]: 2024-09-10 18:48:23.736060214 +0000 UTC m=+0.122981545 container create 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0) 2024-09-10T18:48:24.053 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:48:24.053 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-10T18:48:24.061 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:48:24.061 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:48:24.061 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-10T18:48:24.061 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to enable service . firewalld.service is not available 2024-09-10T18:48:24.069 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:48:24.069 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-10T18:48:24.077 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:48:24.077 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:48:24.077 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-10T18:48:24.078 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-10T18:48:24.078 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr to start... 2024-09-10T18:48:24.078 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr... 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:23 smithi107 podman[29641]: 2024-09-10 18:48:23.938356974 +0000 UTC m=+0.325278260 container init 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:23 smithi107 podman[29641]: 2024-09-10 18:48:23.943105047 +0000 UTC m=+0.330026338 container start 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 bash[29641]: 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 systemd[1]: Started Ceph mgr.y for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.056+0000 7f5052445ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.120+0000 7f5052445ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:48:24.202 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.184+0000 7f5052445ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:48:24.458 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.313+0000 7f5052445ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:48:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/523878596' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:48:24.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.456+0000 7f5052445ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:48:25.093 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.871+0000 7f5052445ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:48:25.093 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:24.931+0000 7f5052445ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:48:25.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:25 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:25.091+0000 7f5052445ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:48:26.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:25 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:25.936+0000 7f5052445ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:48:26.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:26 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:26.001+0000 7f5052445ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:48:26.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:26 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:26.708+0000 7f5052445ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:48:26.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:26 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:26.819+0000 7f5052445ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:48:26.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:26 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:26.942+0000 7f5052445ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:48:27.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.002+0000 7f5052445ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:48:27.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.129+0000 7f5052445ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:48:27.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.196+0000 7f5052445ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:48:27.255 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.253+0000 7f5052445ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:48:27.518 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.389+0000 7f5052445ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:48:27.519 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.451+0000 7f5052445ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:48:27.519 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.517+0000 7f5052445ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:48:27.824 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.598+0000 7f5052445ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:48:27.825 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.657+0000 7f5052445ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:48:27.825 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:27 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:27.713+0000 7f5052445ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:48:28.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:28.466+0000 7f5052445ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: Activating manager daemon y 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: mgrmap e2: y(active, starting, since 0.00292502s) 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: Manager daemon y is now available 2024-09-10T18:48:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:48:28.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:48:28.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' 2024-09-10T18:48:28.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' 2024-09-10T18:48:28.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:28 smithi107 ceph-mon[29404]: from='mgr.14100 172.21.15.107:0/1009930478' entity='mgr.y' 2024-09-10T18:48:30.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:30.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:48:30.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "d7c9fa7c-6fa4-11ef-bcea-c7b262605968", 2024-09-10T18:48:30.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:48:30.672 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-10T18:48:30.673 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:48:30.674 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:48:30.682 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:48:30.683 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:48:17.432234+0000", 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:48:30.684 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.527+0000 7fa6b77db640 1 Processor -- start 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b77db640 1 -- start start 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b77db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b77db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b0103410 con 0x7fa6b0073c80 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b5577640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b5577640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53132/0 (socket says 172.21.15.107:53132) 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.528+0000 7fa6b5577640 1 -- 172.21.15.107:0/245418652 learned_addr learned my addr 172.21.15.107:0/245418652 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.529+0000 7fa6b5577640 1 -- 172.21.15.107:0/245418652 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6b0103550 con 0x7fa6b0073c80 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b5577640 1 --2- 172.21.15.107:0/245418652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009840 tx=0x7fa6ac0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcb6372fcaec52a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6a7fff640 1 -- 172.21.15.107:0/245418652 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac02add0 con 0x7fa6b0073c80 2024-09-10T18:48:30.685 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6a7fff640 1 -- 172.21.15.107:0/245418652 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fa6ac032860 con 0x7fa6b0073c80 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6a7fff640 1 -- 172.21.15.107:0/245418652 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac031630 con 0x7fa6b0073c80 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b77db640 1 -- 172.21.15.107:0/245418652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 msgr2=0x7fa6b0102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b77db640 1 --2- 172.21.15.107:0/245418652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac009840 tx=0x7fa6ac0091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b77db640 1 -- 172.21.15.107:0/245418652 shutdown_connections 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b77db640 1 --2- 172.21.15.107:0/245418652 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b0102ed0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.530+0000 7fa6b77db640 1 -- 172.21.15.107:0/245418652 >> 172.21.15.107:0/245418652 conn(0x7fa6b00faa00 msgr2=0x7fa6b00fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 -- 172.21.15.107:0/245418652 shutdown_connections 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 -- 172.21.15.107:0/245418652 wait complete. 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 Processor -- start 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 -- start start 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b77db640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6b010f420 con 0x7fa6b0073c80 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b5577640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:30.686 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b5577640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53140/0 (socket says 172.21.15.107:53140) 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.531+0000 7fa6b5577640 1 -- 172.21.15.107:0/523878596 learned_addr learned my addr 172.21.15.107:0/523878596 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6b5577640 1 -- 172.21.15.107:0/523878596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6ac009450 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6b5577640 1 --2- 172.21.15.107:0/523878596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0319c0 tx=0x7fa6ac02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac031e50 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7fa6ac03b860 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa6ac03a3a0 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6b010f620 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6b01100d0 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.532+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 1) v1 ==== 529+0+0 (secure 0 0 0) 0x7fa6ac039070 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.533+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7fa6ac03adc0 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.533+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6b010fa90 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.536+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71553 (secure 0 0 0) 0x7fa6ac049060 con 0x7fa6b0073c80 2024-09-10T18:48:30.687 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.596+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7fa6b005ed70 con 0x7fa6b0073c80 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.596+0000 7fa6a5ffb640 1 -- 172.21.15.107:0/523878596 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7fa6ac03f070 con 0x7fa6b0073c80 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 msgr2=0x7fa6b010eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 --2- 172.21.15.107:0/523878596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fa6ac0319c0 tx=0x7fa6ac02c040 comp rx=0 tx=0).stop 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 shutdown_connections 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 --2- 172.21.15.107:0/523878596 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa6b0073c80 0x7fa6b010eee0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 >> 172.21.15.107:0/523878596 conn(0x7fa6b00faa00 msgr2=0x7fa6b00fb3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 shutdown_connections 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:24.597+0000 7fa6b77db640 1 -- 172.21.15.107:0/523878596 wait complete. 2024-09-10T18:48:30.688 INFO:teuthology.orchestra.run.smithi107.stdout:mgr not available, waiting (1/15)... 2024-09-10T18:48:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:30 smithi107 ceph-mon[29404]: mgrmap e3: y(active, since 1.00722s) 2024-09-10T18:48:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:31 smithi107 ceph-mon[29404]: mgrmap e4: y(active, since 2s) 2024-09-10T18:48:33.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1284430098' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-09-10T18:48:33.971 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsid": "d7c9fa7c-6fa4-11ef-bcea-c7b262605968", 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "health": { 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-10T18:48:33.974 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 0 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "a" 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "quorum_age": 11, 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "quincy", 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-10T18:48:33.975 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-10T18:48:33.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:33.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-10T18:48:33.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-10T18:48:33.976 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-10T18:48:33.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-10T18:48:33.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "restful" 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ], 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:48:33.986 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-10T18:48:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-10T18:48:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "modified": "2024-09-10T18:48:17.432234+0000", 2024-09-10T18:48:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-10T18:48:33.987 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout }, 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.090+0000 7f1e7cbdb640 1 Processor -- start 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.090+0000 7f1e7cbdb640 1 -- start start 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.090+0000 7f1e7cbdb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:33.989 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.090+0000 7f1e7cbdb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e78100ca0 con 0x7f1e78100350 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.090+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.091+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53204/0 (socket says 172.21.15.107:53204) 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.091+0000 7f1e7659c640 1 -- 172.21.15.107:0/3651959540 learned_addr learned my addr 172.21.15.107:0/3651959540 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.091+0000 7f1e7659c640 1 -- 172.21.15.107:0/3651959540 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e78100de0 con 0x7f1e78100350 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.091+0000 7f1e7659c640 1 --2- 172.21.15.107:0/3651959540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e50 tx=0x7f1e70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e15602a5263aebf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7559a640 1 -- 172.21.15.107:0/3651959540 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e70031400 con 0x7f1e78100350 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7559a640 1 -- 172.21.15.107:0/3651959540 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f1e7002d050 con 0x7f1e78100350 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7559a640 1 -- 172.21.15.107:0/3651959540 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e70007580 con 0x7f1e78100350 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/3651959540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 msgr2=0x7f1e78100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/3651959540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1e70009e50 tx=0x7f1e70009840 comp rx=0 tx=0).stop 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/3651959540 shutdown_connections 2024-09-10T18:48:33.990 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/3651959540 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e78100760 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/3651959540 >> 172.21.15.107:0/3651959540 conn(0x7f1e780fbb50 msgr2=0x7f1e780fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/3651959540 shutdown_connections 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.092+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/3651959540 wait complete. 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7cbdb640 1 Processor -- start 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7cbdb640 1 -- start start 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7cbdb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7cbdb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7810f400 con 0x7f1e78100350 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53214/0 (socket says 172.21.15.107:53214) 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7659c640 1 -- 172.21.15.107:0/1284430098 learned_addr learned my addr 172.21.15.107:0/1284430098 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7659c640 1 -- 172.21.15.107:0/1284430098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e70009ae0 con 0x7f1e78100350 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.093+0000 7f1e7659c640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e70003400 tx=0x7f1e70009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:33.991 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e70007a90 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f1e70007bf0 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e7810f600 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1e70045d60 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e7810fa70 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.094+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7f1e70038040 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.095+0000 7f1e6effd640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 0x7f1e6403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.095+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f1e70040030 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.095+0000 7f1e75d9b640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 0x7f1e6403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.095+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e7805ed70 con 0x7f1e78100350 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.095+0000 7f1e75d9b640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 0x7f1e6403b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.100+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1e70036070 con 0x7f1e78100350 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.314+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) v1 -- 0x7f1e781021f0 con 0x7f1e78100350 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.314+0000 7f1e6effd640 1 -- 172.21.15.107:0/1284430098 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) v1 ==== 79+0+1240 (secure 0 0 0) 0x7f1e781021f0 con 0x7f1e78100350 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 msgr2=0x7f1e6403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 0x7f1e6403b910 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f1e60006fd0 tx=0x7f1e60006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 msgr2=0x7f1e7810eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1e70003400 tx=0x7f1e70009840 comp rx=0 tx=0).stop 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 shutdown_connections 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f1e64039460 0x7f1e6403b910 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 --2- 172.21.15.107:0/1284430098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1e78100350 0x7f1e7810eec0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 >> 172.21.15.107:0/1284430098 conn(0x7f1e780fbb50 msgr2=0x7f1e780fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 shutdown_connections 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:33.316+0000 7f1e7cbdb640 1 -- 172.21.15.107:0/1284430098 wait complete. 2024-09-10T18:48:33.993 INFO:teuthology.orchestra.run.smithi107.stdout:mgr is available 2024-09-10T18:48:34.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1915216014' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [global] 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout fsid = d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-10T18:48:35.175 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout [osd] 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f9251f0c640 1 Processor -- start 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f9251f0c640 1 -- start start 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f9251f0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f9251f0c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c0ffb60 con 0x7f924c0ff210 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:35.176 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50974/0 (socket says 172.21.15.107:50974) 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.394+0000 7f924b7fe640 1 -- 172.21.15.107:0/3097227924 learned_addr learned my addr 172.21.15.107:0/3097227924 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.395+0000 7f924b7fe640 1 -- 172.21.15.107:0/3097227924 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f924c0ffca0 con 0x7f924c0ff210 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.395+0000 7f924b7fe640 1 --2- 172.21.15.107:0/3097227924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f923c009840 tx=0x7f923c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c5720a8c6e64a93 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f924a7fc640 1 -- 172.21.15.107:0/3097227924 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f923c02add0 con 0x7f924c0ff210 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f924a7fc640 1 -- 172.21.15.107:0/3097227924 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f923c032860 con 0x7f924c0ff210 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f924a7fc640 1 -- 172.21.15.107:0/3097227924 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f923c031630 con 0x7f924c0ff210 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 -- 172.21.15.107:0/3097227924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 msgr2=0x7f924c0ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 --2- 172.21.15.107:0/3097227924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f923c009840 tx=0x7f923c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 -- 172.21.15.107:0/3097227924 shutdown_connections 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 --2- 172.21.15.107:0/3097227924 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c0ff620 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 -- 172.21.15.107:0/3097227924 >> 172.21.15.107:0/3097227924 conn(0x7f924c0faa10 msgr2=0x7f924c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 -- 172.21.15.107:0/3097227924 shutdown_connections 2024-09-10T18:48:35.177 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.396+0000 7f9251f0c640 1 -- 172.21.15.107:0/3097227924 wait complete. 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f9251f0c640 1 Processor -- start 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f9251f0c640 1 -- start start 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f9251f0c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f9251f0c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f924c10d210 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f924b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50976/0 (socket says 172.21.15.107:50976) 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f924b7fe640 1 -- 172.21.15.107:0/1915216014 learned_addr learned my addr 172.21.15.107:0/1915216014 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.397+0000 7f924b7fe640 1 -- 172.21.15.107:0/1915216014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f923c009450 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f924b7fe640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f923c031ad0 tx=0x7f923c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f923c03b440 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f923c03b9c0 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f924c10d410 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f923c03a3a0 con 0x7f924c0ff210 2024-09-10T18:48:35.178 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f924c10d880 con 0x7f924c0ff210 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.398+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7f923c039070 con 0x7f924c0ff210 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.399+0000 7f9233fff640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 0x7f923403dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.399+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f924c04b720 con 0x7f924c0ff210 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.399+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f923c06c510 con 0x7f924c0ff210 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.399+0000 7f924affd640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 0x7f923403dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.399+0000 7f924affd640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 0x7f923403dad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9240006fd0 tx=0x7f9240006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.403+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f923c037070 con 0x7f924c0ff210 2024-09-10T18:48:35.179 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.560+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) v1 -- 0x7f924c10d880 con 0x7f924c0ff210 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.560+0000 7f9233fff640 1 -- 172.21.15.107:0/1915216014 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) v1 ==== 70+0+318 (secure 0 0 0) 0x7f923c068020 con 0x7f924c0ff210 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 msgr2=0x7f923403dad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 0x7f923403dad0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f9240006fd0 tx=0x7f9240006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 msgr2=0x7f924c10ccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f923c031ad0 tx=0x7f923c02c040 comp rx=0 tx=0).stop 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 shutdown_connections 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f923403b620 0x7f923403dad0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 --2- 172.21.15.107:0/1915216014 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f924c0ff210 0x7f924c10ccd0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 >> 172.21.15.107:0/1915216014 conn(0x7f924c0faa10 msgr2=0x7f924c103b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:35.180 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 shutdown_connections 2024-09-10T18:48:35.181 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:34.562+0000 7f9251f0c640 1 -- 172.21.15.107:0/1915216014 wait complete. 2024-09-10T18:48:35.181 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling cephadm module... 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1487938352' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:35 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setuser ceph since I am not root 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:35 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setgroup ceph since I am not root 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:35 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:35.962+0000 7f49cfd2cec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.015+0000 7f49cfd2cec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:48:36.083 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.081+0000 7f49cfd2cec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:48:36.341 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.204+0000 7f49cfd2cec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:48:36.341 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.339+0000 7f49cfd2cec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:48:36.875 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1487938352' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-10T18:48:36.876 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-mon[29404]: mgrmap e5: y(active, since 7s) 2024-09-10T18:48:36.876 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.595+0000 7f49cfd2cec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:48:36.876 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.654+0000 7f49cfd2cec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:48:36.876 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:36 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:36.813+0000 7f49cfd2cec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.571+0000 7f382828a640 1 Processor -- start 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.571+0000 7f382828a640 1 -- start start 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.571+0000 7f382828a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.571+0000 7f382828a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38200ffb30 con 0x7f38200ff1e0 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.572+0000 7f3826026640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.572+0000 7f3826026640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50988/0 (socket says 172.21.15.107:50988) 2024-09-10T18:48:37.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.572+0000 7f3826026640 1 -- 172.21.15.107:0/1575489675 learned_addr learned my addr 172.21.15.107:0/1575489675 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.572+0000 7f3826026640 1 -- 172.21.15.107:0/1575489675 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38200ffc70 con 0x7f38200ff1e0 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.572+0000 7f3826026640 1 --2- 172.21.15.107:0/1575489675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3810009e50 tx=0x7f3810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c48836e4838eeb3 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f3825024640 1 -- 172.21.15.107:0/1575489675 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810031400 con 0x7f38200ff1e0 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f3825024640 1 -- 172.21.15.107:0/1575489675 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f381002d050 con 0x7f38200ff1e0 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f3825024640 1 -- 172.21.15.107:0/1575489675 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810007580 con 0x7f38200ff1e0 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 -- 172.21.15.107:0/1575489675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 msgr2=0x7f38200ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 --2- 172.21.15.107:0/1575489675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f3810009e50 tx=0x7f3810009840 comp rx=0 tx=0).stop 2024-09-10T18:48:37.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 -- 172.21.15.107:0/1575489675 shutdown_connections 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 --2- 172.21.15.107:0/1575489675 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f38200ff5f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 -- 172.21.15.107:0/1575489675 >> 172.21.15.107:0/1575489675 conn(0x7f38200faa00 msgr2=0x7f38200fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 -- 172.21.15.107:0/1575489675 shutdown_connections 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.573+0000 7f382828a640 1 -- 172.21.15.107:0/1575489675 wait complete. 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f382828a640 1 Processor -- start 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f382828a640 1 -- start start 2024-09-10T18:48:37.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f382828a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f382828a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3820111bc0 con 0x7f38200ff1e0 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f3826026640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f3826026640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50998/0 (socket says 172.21.15.107:50998) 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f3826026640 1 -- 172.21.15.107:0/1487938352 learned_addr learned my addr 172.21.15.107:0/1487938352 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f3826026640 1 -- 172.21.15.107:0/1487938352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3810009ae0 con 0x7f38200ff1e0 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.574+0000 7f3826026640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3810003400 tx=0x7f3810009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.575+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810007a90 con 0x7f38200ff1e0 2024-09-10T18:48:37.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.575+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f3810007bf0 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.575+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3820111d00 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.575+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f3810045d60 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.575+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3820112110 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.576+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 4) v1 ==== 45481+0+0 (secure 0 0 0) 0x7f3810038040 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.576+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f382005ed70 con 0x7f38200ff1e0 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.576+0000 7f381effd640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 0x7f380803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:37.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.576+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3810040030 con 0x7f38200ff1e0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.576+0000 7f3825825640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 0x7f380803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.577+0000 7f3825825640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 0x7f380803b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3814006fd0 tx=0x7f3814006db0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.581+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f381003a400 con 0x7f38200ff1e0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.778+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) v1 -- 0x7f382004b720 con 0x7f38200ff1e0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.831+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 5) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f381004c300 con 0x7f38200ff1e0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.832+0000 7f381effd640 1 -- 172.21.15.107:0/1487938352 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) v1 ==== 86+0+0 (secure 0 0 0) 0x7f381002f950 con 0x7f38200ff1e0 2024-09-10T18:48:37.458 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 msgr2=0x7f380803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 0x7f380803b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3814006fd0 tx=0x7f3814006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 msgr2=0x7f3820073ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f3810003400 tx=0x7f3810009840 comp rx=0 tx=0).stop 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 shutdown_connections 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:6800/1592664852,v1:172.21.15.107:6801/1592664852] conn(0x7f3808039420 0x7f380803b8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 --2- 172.21.15.107:0/1487938352 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38200ff1e0 0x7f3820073ae0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 >> 172.21.15.107:0/1487938352 conn(0x7f38200faa00 msgr2=0x7f38200fb410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 shutdown_connections 2024-09-10T18:48:37.459 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:35.834+0000 7f382828a640 1 -- 172.21.15.107:0/1487938352 wait complete. 2024-09-10T18:48:37.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:37 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:37.701+0000 7f49cfd2cec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:48:37.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:37 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:37.767+0000 7f49cfd2cec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:48:38.767 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.475+0000 7f49cfd2cec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:48:38.767 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.586+0000 7f49cfd2cec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:48:38.768 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.707+0000 7f49cfd2cec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:48:38.768 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.765+0000 7f49cfd2cec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:48:39.020 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.894+0000 7f49cfd2cec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:48:39.020 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:38 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:38.960+0000 7f49cfd2cec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:48:39.020 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.018+0000 7f49cfd2cec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:48:39.291 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.154+0000 7f49cfd2cec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:48:39.291 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.218+0000 7f49cfd2cec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:48:39.291 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.289+0000 7f49cfd2cec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:48:39.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.373+0000 7f49cfd2cec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:48:39.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.429+0000 7f49cfd2cec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:48:39.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:39.485+0000 7f49cfd2cec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:48:40.574 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:40.229+0000 7f49cfd2cec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: Active manager daemon y restarted 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: Activating manager daemon y 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: osdmap e2: 0 total, 0 up, 0 in 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: mgrmap e6: y(active, starting, since 0.00283241s) 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: Manager daemon y is now available 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:40.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:40.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:40.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:48:40.576 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:40 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:48:41.195 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 6, 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f704df4c640 1 Processor -- start 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f704df4c640 1 -- start start 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f704df4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f704df4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048103460 con 0x7f70480744c0 2024-09-10T18:48:41.196 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f70477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f70477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51110/0 (socket says 172.21.15.107:51110) 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.362+0000 7f70477fe640 1 -- 172.21.15.107:0/779604357 learned_addr learned my addr 172.21.15.107:0/779604357 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.363+0000 7f70477fe640 1 -- 172.21.15.107:0/779604357 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70481035a0 con 0x7f70480744c0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.363+0000 7f70477fe640 1 --2- 172.21.15.107:0/779604357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7038009840 tx=0x7f70380091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c9df6c27b23fdf server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.363+0000 7f70467fc640 1 -- 172.21.15.107:0/779604357 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f703802add0 con 0x7f70480744c0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.363+0000 7f70467fc640 1 -- 172.21.15.107:0/779604357 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f7038032860 con 0x7f70480744c0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.363+0000 7f70467fc640 1 -- 172.21.15.107:0/779604357 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7038031630 con 0x7f70480744c0 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 -- 172.21.15.107:0/779604357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 msgr2=0x7f7048102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 --2- 172.21.15.107:0/779604357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7038009840 tx=0x7f70380091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 -- 172.21.15.107:0/779604357 shutdown_connections 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 --2- 172.21.15.107:0/779604357 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048102f20 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:41.197 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 -- 172.21.15.107:0/779604357 >> 172.21.15.107:0/779604357 conn(0x7f70480faa30 msgr2=0x7f70480fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 -- 172.21.15.107:0/779604357 shutdown_connections 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.364+0000 7f704df4c640 1 -- 172.21.15.107:0/779604357 wait complete. 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f704df4c640 1 Processor -- start 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f704df4c640 1 -- start start 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f704df4c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f704df4c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7048100d90 con 0x7f70480744c0 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f70477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f70477fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51122/0 (socket says 172.21.15.107:51122) 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f70477fe640 1 -- 172.21.15.107:0/1813391165 learned_addr learned my addr 172.21.15.107:0/1813391165 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f70477fe640 1 -- 172.21.15.107:0/1813391165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7038009450 con 0x7f70480744c0 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.365+0000 7f70477fe640 1 --2- 172.21.15.107:0/1813391165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7038006a70 tx=0x7f703802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.366+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7038031d00 con 0x7f70480744c0 2024-09-10T18:48:41.198 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.366+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f7038031e60 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.366+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70480fee20 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.366+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f703803a3a0 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.366+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70480ff290 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.367+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 6) v1 ==== 45259+0+0 (secure 0 0 0) 0x7f7038039070 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.367+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f704805ed70 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.367+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f703806c450 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.376+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f703803f080 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.572+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f70481041b0 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.573+0000 7f702ffff640 1 -- 172.21.15.107:0/1813391165 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v6) v1 ==== 56+0+89 (secure 0 0 0) 0x7f7038037070 con 0x7f70480744c0 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 msgr2=0x7f7048100850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 --2- 172.21.15.107:0/1813391165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7038006a70 tx=0x7f703802c040 comp rx=0 tx=0).stop 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 shutdown_connections 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 --2- 172.21.15.107:0/1813391165 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f70480744c0 0x7f7048100850 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:41.199 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 >> 172.21.15.107:0/1813391165 conn(0x7f70480faa30 msgr2=0x7f70480fb560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:41.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 shutdown_connections 2024-09-10T18:48:41.200 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:40.575+0000 7f704df4c640 1 -- 172.21.15.107:0/1813391165 wait complete. 2024-09-10T18:48:41.200 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-10T18:48:41.200 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 6... 2024-09-10T18:48:41.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-mon[29404]: Found migration_current of "None". Setting to last migration. 2024-09-10T18:48:41.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1813391165' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:48:41.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-mon[29404]: mgrmap e7: y(active, since 1.00464s) 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:41.504+0000 7f498211a640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:48:41] ENGINE Error in HTTPServer.serve 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Traceback (most recent call last): 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._connections.run(self.expiration_interval) 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:48:41.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._run(expiration_interval) 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s = self.context.wrap_socket( 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: return self.sslsocket_class._create( 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self.do_handshake() 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._sslobj.do_handshake() 2024-09-10T18:48:41.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:48:41.718 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:41 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.645+0000 7feba49e3640 1 Processor -- start 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.645+0000 7feba49e3640 1 -- start start 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.645+0000 7feba49e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.645+0000 7feba49e3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba0100c60 con 0x7feba0074530 2024-09-10T18:48:42.223 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.645+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.646+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51136/0 (socket says 172.21.15.107:51136) 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.646+0000 7feb9e59c640 1 -- 172.21.15.107:0/1419439040 learned_addr learned my addr 172.21.15.107:0/1419439040 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.646+0000 7feb9e59c640 1 -- 172.21.15.107:0/1419439040 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba01057b0 con 0x7feba0074530 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.646+0000 7feb9e59c640 1 --2- 172.21.15.107:0/1419439040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb90009840 tx=0x7feb900091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf98cb67b24427d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feb9d59a640 1 -- 172.21.15.107:0/1419439040 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb9002add0 con 0x7feba0074530 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feb9d59a640 1 -- 172.21.15.107:0/1419439040 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7feb90032860 con 0x7feba0074530 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feb9d59a640 1 -- 172.21.15.107:0/1419439040 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb90031630 con 0x7feba0074530 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 -- 172.21.15.107:0/1419439040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 msgr2=0x7feba0105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 --2- 172.21.15.107:0/1419439040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb90009840 tx=0x7feb900091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 -- 172.21.15.107:0/1419439040 shutdown_connections 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 --2- 172.21.15.107:0/1419439040 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0105270 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 -- 172.21.15.107:0/1419439040 >> 172.21.15.107:0/1419439040 conn(0x7feba00fa9e0 msgr2=0x7feba00fcdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:42.224 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 -- 172.21.15.107:0/1419439040 shutdown_connections 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.647+0000 7feba49e3640 1 -- 172.21.15.107:0/1419439040 wait complete. 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feba49e3640 1 Processor -- start 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feba49e3640 1 -- start start 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feba49e3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feba49e3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba01115b0 con 0x7feba0074530 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feb9e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51144/0 (socket says 172.21.15.107:51144) 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feb9e59c640 1 -- 172.21.15.107:0/1811939579 learned_addr learned my addr 172.21.15.107:0/1811939579 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feb9e59c640 1 -- 172.21.15.107:0/1811939579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb90009450 con 0x7feba0074530 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.648+0000 7feb9e59c640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb900319c0 tx=0x7feb9002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb90031e50 con 0x7feba0074530 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7feb9003b860 con 0x7feba0074530 2024-09-10T18:48:42.225 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba01117b0 con 0x7feba0074530 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7feb9003a3a0 con 0x7feba0074530 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba0111c20 con 0x7feba0074530 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.649+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 7) v1 ==== 45386+0+0 (secure 0 0 0) 0x7feb90039070 con 0x7feba0074530 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb8effd640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 0x7feb8803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7feb8803bf50 con 0x7feb88039390 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7feb9006c440 con 0x7feba0074530 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb9dd9b640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 0x7feb8803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb9dd9b640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 0x7feb8803b840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.650+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7feb8803bf50 con 0x7feb88039390 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.655+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7feba0111c20 con 0x7feb88039390 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.655+0000 7feb8effd640 1 -- 172.21.15.107:0/1811939579 <== mgr.14116 v2:172.21.15.107:6800/2559986261 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7feba0111c20 con 0x7feb88039390 2024-09-10T18:48:42.226 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 msgr2=0x7feb8803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 0x7feb8803b840 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7feb94006fd0 tx=0x7feb94006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 msgr2=0x7feba0111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7feb900319c0 tx=0x7feb9002c040 comp rx=0 tx=0).stop 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 shutdown_connections 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7feb88039390 0x7feb8803b840 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 --2- 172.21.15.107:0/1811939579 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7feba0074530 0x7feba0111070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 >> 172.21.15.107:0/1811939579 conn(0x7feba00fa9e0 msgr2=0x7feba00fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 shutdown_connections 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:41.656+0000 7feba49e3640 1 -- 172.21.15.107:0/1811939579 wait complete. 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 6 is available 2024-09-10T18:48:42.227 INFO:teuthology.orchestra.run.smithi107.stdout:Setting orchestrator backend to cephadm... 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: [10/Sep/2024:18:48:41] ENGINE Bus STARTING 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: [10/Sep/2024:18:48:41] ENGINE Serving on https://172.21.15.107:7150 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: [10/Sep/2024:18:48:41] ENGINE Bus STARTED 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: [10/Sep/2024:18:48:41] ENGINE Error in HTTPServer.serve 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-10T18:48:42.602 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:48:42.603 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-10T18:48:42.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:42 smithi107 ceph-mon[29404]: from='client.14126 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c14854640 1 Processor -- start 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c14854640 1 -- start start 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c14854640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c14854640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10076950 con 0x7f5c10076540 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:43.463 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51150/0 (socket says 172.21.15.107:51150) 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c0e59c640 1 -- 172.21.15.107:0/304210898 learned_addr learned my addr 172.21.15.107:0/304210898 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.677+0000 7f5c0e59c640 1 -- 172.21.15.107:0/304210898 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c10075000 con 0x7f5c10076540 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.678+0000 7f5c0e59c640 1 --2- 172.21.15.107:0/304210898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009840 tx=0x7f5c000091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc61174ea303ca7 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.678+0000 7f5c0d59a640 1 -- 172.21.15.107:0/304210898 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0002add0 con 0x7f5c10076540 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.678+0000 7f5c0d59a640 1 -- 172.21.15.107:0/304210898 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f5c00032860 con 0x7f5c10076540 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.678+0000 7f5c0d59a640 1 -- 172.21.15.107:0/304210898 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c00031630 con 0x7f5c10076540 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 -- 172.21.15.107:0/304210898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 msgr2=0x7f5c10074ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 --2- 172.21.15.107:0/304210898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f5c00009840 tx=0x7f5c000091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:43.464 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 -- 172.21.15.107:0/304210898 shutdown_connections 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 --2- 172.21.15.107:0/304210898 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c10074ac0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 -- 172.21.15.107:0/304210898 >> 172.21.15.107:0/304210898 conn(0x7f5c100fbb50 msgr2=0x7f5c100fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 -- 172.21.15.107:0/304210898 shutdown_connections 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 -- 172.21.15.107:0/304210898 wait complete. 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.679+0000 7f5c14854640 1 Processor -- start 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c14854640 1 -- start start 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c14854640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c14854640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c10111630 con 0x7f5c10076540 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51156/0 (socket says 172.21.15.107:51156) 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c0e59c640 1 -- 172.21.15.107:0/3909502100 learned_addr learned my addr 172.21.15.107:0/3909502100 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c0e59c640 1 -- 172.21.15.107:0/3909502100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c00009450 con 0x7f5c10076540 2024-09-10T18:48:43.465 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c0e59c640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006a70 tx=0x7f5c0002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:43.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0003b440 con 0x7f5c10076540 2024-09-10T18:48:43.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f5c0003b9c0 con 0x7f5c10076540 2024-09-10T18:48:43.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c10111830 con 0x7f5c10076540 2024-09-10T18:48:43.466 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f5c0003a3a0 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.680+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c10111ca0 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.681+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c1004b720 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.682+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f5c00039070 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.682+0000 7f5bfeffd640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 0x7f5bf803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.682+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f5c0006c780 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.682+0000 7f5c0dd9b640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 0x7f5bf803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.682+0000 7f5c0dd9b640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 0x7f5bf803b8d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.686+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5c00039920 con 0x7f5c10076540 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.853+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) v1 -- 0x7f5c10111ca0 con 0x7f5bf8039420 2024-09-10T18:48:43.467 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.862+0000 7f5bfeffd640 1 -- 172.21.15.107:0/3909502100 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f5c10111ca0 con 0x7f5bf8039420 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 msgr2=0x7f5bf803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 0x7f5bf803b8d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f5c04006fd0 tx=0x7f5c04006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 msgr2=0x7f5c101110f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f5c00006a70 tx=0x7f5c0002c040 comp rx=0 tx=0).stop 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 shutdown_connections 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f5bf8039420 0x7f5bf803b8d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 --2- 172.21.15.107:0/3909502100 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5c10076540 0x7f5c101110f0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 >> 172.21.15.107:0/3909502100 conn(0x7f5c100fbb50 msgr2=0x7f5c100fc5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.864+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 shutdown_connections 2024-09-10T18:48:43.468 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:42.865+0000 7f5c14854640 1 -- 172.21.15.107:0/3909502100 wait complete. 2024-09-10T18:48:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:43 smithi107 ceph-mon[29404]: mgrmap e8: y(active, since 2s) 2024-09-10T18:48:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:43 smithi107 ceph-mon[29404]: from='client.14128 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:43 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:43 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008f053640 1 Processor -- start 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008f053640 1 -- start start 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008f053640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008f053640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0088102160 con 0x7f0088101780 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008cdef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008cdef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51158/0 (socket says 172.21.15.107:51158) 2024-09-10T18:48:45.062 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.866+0000 7f008cdef640 1 -- 172.21.15.107:0/510239342 learned_addr learned my addr 172.21.15.107:0/510239342 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.867+0000 7f008cdef640 1 -- 172.21.15.107:0/510239342 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00881022a0 con 0x7f0088101780 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.867+0000 7f008cdef640 1 --2- 172.21.15.107:0/510239342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0080009840 tx=0x7f00800091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3721cbe4bb1bdb0c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.867+0000 7f00877fe640 1 -- 172.21.15.107:0/510239342 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f008002add0 con 0x7f0088101780 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.867+0000 7f00877fe640 1 -- 172.21.15.107:0/510239342 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f0080032860 con 0x7f0088101780 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.867+0000 7f00877fe640 1 -- 172.21.15.107:0/510239342 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0080031630 con 0x7f0088101780 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 -- 172.21.15.107:0/510239342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 msgr2=0x7f0088101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 --2- 172.21.15.107:0/510239342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0080009840 tx=0x7f00800091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 -- 172.21.15.107:0/510239342 shutdown_connections 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 --2- 172.21.15.107:0/510239342 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f0088101b90 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 -- 172.21.15.107:0/510239342 >> 172.21.15.107:0/510239342 conn(0x7f00880fa150 msgr2=0x7f00880fc560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 -- 172.21.15.107:0/510239342 shutdown_connections 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.868+0000 7f008f053640 1 -- 172.21.15.107:0/510239342 wait complete. 2024-09-10T18:48:45.063 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008f053640 1 Processor -- start 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008f053640 1 -- start start 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008f053640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008f053640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f008810f440 con 0x7f0088101780 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008cdef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008cdef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51174/0 (socket says 172.21.15.107:51174) 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008cdef640 1 -- 172.21.15.107:0/2451730088 learned_addr learned my addr 172.21.15.107:0/2451730088 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008cdef640 1 -- 172.21.15.107:0/2451730088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0080009450 con 0x7f0088101780 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.869+0000 7f008cdef640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0080031ad0 tx=0x7f008002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f008003b440 con 0x7f0088101780 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f008003b9c0 con 0x7f0088101780 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f008810f640 con 0x7f0088101780 2024-09-10T18:48:45.064 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f008003a3a0 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00881100f0 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f0080039070 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.870+0000 7f00857fa640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 0x7f007403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.871+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f008006c950 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.871+0000 7f0087fff640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 0x7f007403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.871+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f008810fab0 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.871+0000 7f0087fff640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 0x7f007403b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f007800ad90 tx=0x7f0078009410 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:43.875+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f008003f080 con 0x7f0088101780 2024-09-10T18:48:45.065 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.032+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) v1 -- 0x7f0088110330 con 0x7f0074039420 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.032+0000 7f00857fa640 1 -- 172.21.15.107:0/2451730088 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+16 (secure 0 0 0) 0x7f0088110330 con 0x7f0074039420 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 msgr2=0x7f007403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 0x7f007403b8d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f007800ad90 tx=0x7f0078009410 comp rx=0 tx=0).stop 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 msgr2=0x7f008810ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0080031ad0 tx=0x7f008002c040 comp rx=0 tx=0).stop 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 shutdown_connections 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f0074039420 0x7f007403b8d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 --2- 172.21.15.107:0/2451730088 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0088101780 0x7f008810ef00 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 >> 172.21.15.107:0/2451730088 conn(0x7f00880fa150 msgr2=0x7f0088105d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 shutdown_connections 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:44.034+0000 7f008f053640 1 -- 172.21.15.107:0/2451730088 wait complete. 2024-09-10T18:48:45.066 INFO:teuthology.orchestra.run.smithi107.stdout:Generating ssh key... 2024-09-10T18:48:45.184 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:44 smithi107 ceph-mon[29404]: from='client.14130 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Generating public/private rsa key pair. 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Your identification has been saved in /tmp/tmpgq6usi6c/key 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Your public key has been saved in /tmp/tmpgq6usi6c/key.pub 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: The key fingerprint is: 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: SHA256:Ja8ZWflj6x3/ThUBjUVc34heYAFfkqsK8a1QKzoMDpg ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: The key's randomart image is: 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: +---[RSA 3072]----+ 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | ..=+O+o| 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | +.* ++| 2024-09-10T18:48:45.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | . + o.o.o| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | . .* o.. .| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: |.. +So..= .| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: |E . + o+o. o .| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | o o . +oo . . .| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | . + o . . + | 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: | . . ..=| 2024-09-10T18:48:45.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:45 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: +----[SHA256]-----+ 2024-09-10T18:48:46.283 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ef1b640 1 Processor -- start 2024-09-10T18:48:46.283 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ef1b640 1 -- start start 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ef1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ef1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381022f0 con 0x7f9d381019a0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ccb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ccb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35996/0 (socket says 172.21.15.107:35996) 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ccb7640 1 -- 172.21.15.107:0/4232677506 learned_addr learned my addr 172.21.15.107:0/4232677506 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.475+0000 7f9d3ccb7640 1 -- 172.21.15.107:0/4232677506 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d38102430 con 0x7f9d381019a0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.476+0000 7f9d3ccb7640 1 --2- 172.21.15.107:0/4232677506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009840 tx=0x7f9d2c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=577a28634bfbf797 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.476+0000 7f9d377fe640 1 -- 172.21.15.107:0/4232677506 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d2c02add0 con 0x7f9d381019a0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.476+0000 7f9d377fe640 1 -- 172.21.15.107:0/4232677506 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f9d2c032860 con 0x7f9d381019a0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.476+0000 7f9d377fe640 1 -- 172.21.15.107:0/4232677506 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d2c031630 con 0x7f9d381019a0 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/4232677506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 msgr2=0x7f9d38101db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:46.284 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/4232677506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c009840 tx=0x7f9d2c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/4232677506 shutdown_connections 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/4232677506 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d38101db0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/4232677506 >> 172.21.15.107:0/4232677506 conn(0x7f9d380fa180 msgr2=0x7f9d380fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/4232677506 shutdown_connections 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/4232677506 wait complete. 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 Processor -- start 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.477+0000 7f9d3ef1b640 1 -- start start 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ef1b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ef1b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d381015f0 con 0x7f9d381019a0 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ccb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ccb7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36008/0 (socket says 172.21.15.107:36008) 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ccb7640 1 -- 172.21.15.107:0/718805419 learned_addr learned my addr 172.21.15.107:0/718805419 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:46.285 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ccb7640 1 -- 172.21.15.107:0/718805419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d2c009450 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d3ccb7640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c031ad0 tx=0x7f9d2c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d2c032ca0 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.478+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f9d2c03b860 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.479+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d380ff680 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.479+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9d2c03a3a0 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.479+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d380ffaf0 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.479+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f9d2c039070 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.480+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d3805ed70 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.480+0000 7f9d357fa640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 0x7f9d2803b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.480+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f9d2c06c8b0 con 0x7f9d381019a0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.480+0000 7f9d37fff640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 0x7f9d2803b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.480+0000 7f9d37fff640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 0x7f9d2803b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d24006fd0 tx=0x7f9d24006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:46.286 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.484+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9d2c030570 con 0x7f9d381019a0 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.640+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d381086b0 con 0x7f9d280394b0 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.688+0000 7f9d357fa640 1 -- 172.21.15.107:0/718805419 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7f9d381086b0 con 0x7f9d280394b0 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 msgr2=0x7f9d2803b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 0x7f9d2803b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f9d24006fd0 tx=0x7f9d24006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 msgr2=0x7f9d381010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f9d2c031ad0 tx=0x7f9d2c02c040 comp rx=0 tx=0).stop 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 shutdown_connections 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f9d280394b0 0x7f9d2803b960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 --2- 172.21.15.107:0/718805419 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9d381019a0 0x7f9d381010b0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.690+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 >> 172.21.15.107:0/718805419 conn(0x7f9d380fa180 msgr2=0x7f9d38105da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.691+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 shutdown_connections 2024-09-10T18:48:46.287 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:45.691+0000 7f9d3ef1b640 1 -- 172.21.15.107:0/718805419 wait complete. 2024-09-10T18:48:46.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:46 smithi107 ceph-mon[29404]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:46.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:46 smithi107 ceph-mon[29404]: Generating ssh key... 2024-09-10T18:48:46.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:46 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:46.846 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:46 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC15SS9qEhoQcFBfLFwBnnu0rDpLFIJ4fqADixT0O7G3EWE3XFNQmXnb5DWANLlMzPzcOUoMuHOAzsglymZGYQMzl5Q/Z2j5sAM2h+s/SpUwFNIDCBk1z4lS3oMYFi8iMgvCZSOR2FicWv6IPIgjvFZrRKnV/oeDOgyylFewipR6UjQthp5hK+RtnnaNFO/AAG7jHmOCDZgstpPFA8jxYmBDC9SWHSSPoUpDu3REfEGKMRO3AdyboNxK30FcfkjH2A3a4+0KPGeZO7stGpLXAacIcZ/E8aRJFpbuhG7XEMzq3ZMokfTv6vYv2T5DxUtphq/pD5UxLR2YLOY+qRqK4L7ulA9iS+9WyuOb6LAtQWlYCk0MMo7YVpHJUPKqEbHvuIdluygNGo3BHZq4/bhRJyxVpWE58y2Awd3rfQeADnQ+stKGVjgxT+DkLEwK16IRR/qFRQZ5Y0qeSHomKkcXH9aCrutXd10WWS7SZJKWr+NDqk222lRehL9Gmu5i4Y2yek= ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13b6905640 1 Processor -- start 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13b6905640 1 -- start start 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13b6905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13b6905640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b01034a0 con 0x7f13b00744b0 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36014/0 (socket says 172.21.15.107:36014) 2024-09-10T18:48:47.357 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.669+0000 7f13affff640 1 -- 172.21.15.107:0/132596884 learned_addr learned my addr 172.21.15.107:0/132596884 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.670+0000 7f13affff640 1 -- 172.21.15.107:0/132596884 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13b01035e0 con 0x7f13b00744b0 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.670+0000 7f13affff640 1 --2- 172.21.15.107:0/132596884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009840 tx=0x7f13a00091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2dd8c2852ff525 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.670+0000 7f13aeffd640 1 -- 172.21.15.107:0/132596884 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a002add0 con 0x7f13b00744b0 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.670+0000 7f13aeffd640 1 -- 172.21.15.107:0/132596884 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f13a0032860 con 0x7f13b00744b0 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.670+0000 7f13aeffd640 1 -- 172.21.15.107:0/132596884 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a0031630 con 0x7f13b00744b0 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 -- 172.21.15.107:0/132596884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 msgr2=0x7f13b0102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 --2- 172.21.15.107:0/132596884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f13a0009840 tx=0x7f13a00091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 -- 172.21.15.107:0/132596884 shutdown_connections 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 --2- 172.21.15.107:0/132596884 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b0102f60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 -- 172.21.15.107:0/132596884 >> 172.21.15.107:0/132596884 conn(0x7f13b00faa10 msgr2=0x7f13b00fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 -- 172.21.15.107:0/132596884 shutdown_connections 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 -- 172.21.15.107:0/132596884 wait complete. 2024-09-10T18:48:47.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.671+0000 7f13b6905640 1 Processor -- start 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13b6905640 1 -- start start 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13b6905640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13b6905640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13b010f410 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13affff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36018/0 (socket says 172.21.15.107:36018) 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13affff640 1 -- 172.21.15.107:0/1505008571 learned_addr learned my addr 172.21.15.107:0/1505008571 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13affff640 1 -- 172.21.15.107:0/1505008571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13a0009450 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.672+0000 7f13affff640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13a0006a70 tx=0x7f13a002c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a003b440 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f13a003b9c0 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13b010f610 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f13a003a3a0 con 0x7f13b00744b0 2024-09-10T18:48:47.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13b010fa80 con 0x7f13b00744b0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.673+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f13a0039070 con 0x7f13b00744b0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.674+0000 7f13acff9640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 0x7f139803b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.674+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f13a006c790 con 0x7f13b00744b0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.674+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13b005ed70 con 0x7f13b00744b0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.674+0000 7f13af7fe640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 0x7f139803b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.674+0000 7f13af7fe640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 0x7f139803b8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.678+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f13a0037070 con 0x7f13b00744b0 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.833+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) v1 -- 0x7f13b00fbc00 con 0x7f1398039420 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.834+0000 7f13acff9640 1 -- 172.21.15.107:0/1505008571 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+595 (secure 0 0 0) 0x7f13b00fbc00 con 0x7f1398039420 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 msgr2=0x7f139803b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:47.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 0x7f139803b8d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f13a4006fd0 tx=0x7f13a4006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 msgr2=0x7f13b010eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f13a0006a70 tx=0x7f13a002c040 comp rx=0 tx=0).stop 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 shutdown_connections 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f1398039420 0x7f139803b8d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 --2- 172.21.15.107:0/1505008571 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f13b00744b0 0x7f13b010eed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 >> 172.21.15.107:0/1505008571 conn(0x7f13b00faa10 msgr2=0x7f13b00fb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 shutdown_connections 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:46.836+0000 7f13b6905640 1 -- 172.21.15.107:0/1505008571 wait complete. 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-10T18:48:47.361 INFO:teuthology.orchestra.run.smithi107.stdout:Adding key to root@localhost authorized_keys... 2024-09-10T18:48:47.362 INFO:teuthology.orchestra.run.smithi107.stdout:Adding host smithi107... 2024-09-10T18:48:47.743 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:47 smithi107 ceph-mon[29404]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:49.175 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:48 smithi107 ceph-mon[29404]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:49.175 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:48 smithi107 ceph-mon[29404]: Deploying cephadm binary to smithi107 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Added host 'smithi107' with addr '172.21.15.107' 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.818+0000 7f7110760640 1 Processor -- start 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f7110760640 1 -- start start 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f7110760640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f7110760640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7108103480 con 0x7f71080744d0 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f710e4fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:51.183 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f710e4fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36028/0 (socket says 172.21.15.107:36028) 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f710e4fc640 1 -- 172.21.15.107:0/1849955826 learned_addr learned my addr 172.21.15.107:0/1849955826 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.819+0000 7f710e4fc640 1 -- 172.21.15.107:0/1849955826 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71081035c0 con 0x7f71080744d0 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.820+0000 7f710e4fc640 1 --2- 172.21.15.107:0/1849955826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f71040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c8dd29473c18d6d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.820+0000 7f710d4fa640 1 -- 172.21.15.107:0/1849955826 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f710402add0 con 0x7f71080744d0 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.820+0000 7f710d4fa640 1 -- 172.21.15.107:0/1849955826 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f7104032860 con 0x7f71080744d0 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.820+0000 7f710d4fa640 1 -- 172.21.15.107:0/1849955826 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f7104031630 con 0x7f71080744d0 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 -- 172.21.15.107:0/1849955826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 msgr2=0x7f7108102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 --2- 172.21.15.107:0/1849955826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7104009840 tx=0x7f71040091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 -- 172.21.15.107:0/1849955826 shutdown_connections 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 --2- 172.21.15.107:0/1849955826 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f7108102f40 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 -- 172.21.15.107:0/1849955826 >> 172.21.15.107:0/1849955826 conn(0x7f71080faa10 msgr2=0x7f71080fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:51.184 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 -- 172.21.15.107:0/1849955826 shutdown_connections 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 -- 172.21.15.107:0/1849955826 wait complete. 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.821+0000 7f7110760640 1 Processor -- start 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f7110760640 1 -- start start 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f7110760640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f7110760640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f710810f410 con 0x7f71080744d0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f710e4fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f710e4fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36036/0 (socket says 172.21.15.107:36036) 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f710e4fc640 1 -- 172.21.15.107:0/896954313 learned_addr learned my addr 172.21.15.107:0/896954313 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f710e4fc640 1 -- 172.21.15.107:0/896954313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7104009450 con 0x7f71080744d0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f710e4fc640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7104006a70 tx=0x7f710402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f710403b440 con 0x7f71080744d0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f710403b9c0 con 0x7f71080744d0 2024-09-10T18:48:51.185 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f710810f610 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f710403a3a0 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.822+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f710810fa80 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.823+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f7104039070 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.823+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f710804b720 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.823+0000 7f70feffd640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 0x7f70f403b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.823+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f710406c910 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.824+0000 7f710dcfb640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 0x7f70f403b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.824+0000 7f710dcfb640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 0x7f70f403b8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006fd0 tx=0x7f70f8006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.828+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7104039790 con 0x7f71080744d0 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:47.988+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi107", "addr": "172.21.15.107", "target": ["mon-mgr", ""]}) v1 -- 0x7f710810fa80 con 0x7f70f4039420 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.596+0000 7f70feffd640 1 -- 172.21.15.107:0/896954313 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f710810fa80 con 0x7f70f4039420 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.598+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 msgr2=0x7f70f403b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:51.186 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.598+0000 7f7110760640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 0x7f70f403b8d0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f70f8006fd0 tx=0x7f70f8006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.598+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 msgr2=0x7f710810eed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.598+0000 7f7110760640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7104006a70 tx=0x7f710402c040 comp rx=0 tx=0).stop 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 shutdown_connections 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f70f4039420 0x7f70f403b8d0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 --2- 172.21.15.107:0/896954313 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f71080744d0 0x7f710810eed0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 >> 172.21.15.107:0/896954313 conn(0x7f71080faa10 msgr2=0x7f71080fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 shutdown_connections 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:50.599+0000 7f7110760640 1 -- 172.21.15.107:0/896954313 wait complete. 2024-09-10T18:48:51.187 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mon service... 2024-09-10T18:48:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:51 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:51 smithi107 ceph-mon[29404]: Added host smithi107 2024-09-10T18:48:51.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:51 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:48:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-10T18:48:52.603 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.714+0000 7f0847534640 1 Processor -- start 2024-09-10T18:48:52.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f0847534640 1 -- start start 2024-09-10T18:48:52.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f0847534640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:52.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f0847534640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08400ffb30 con 0x7f08400ff1e0 2024-09-10T18:48:52.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f08452d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:52.604 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f08452d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36052/0 (socket says 172.21.15.107:36052) 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f08452d0640 1 -- 172.21.15.107:0/964332476 learned_addr learned my addr 172.21.15.107:0/964332476 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.715+0000 7f08452d0640 1 -- 172.21.15.107:0/964332476 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08400ffc70 con 0x7f08400ff1e0 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.716+0000 7f08452d0640 1 --2- 172.21.15.107:0/964332476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f083c009840 tx=0x7f083c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87c75c5f43e7a747 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.716+0000 7f0837fff640 1 -- 172.21.15.107:0/964332476 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f083c02add0 con 0x7f08400ff1e0 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.716+0000 7f0837fff640 1 -- 172.21.15.107:0/964332476 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f083c032860 con 0x7f08400ff1e0 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.716+0000 7f0837fff640 1 -- 172.21.15.107:0/964332476 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f083c031630 con 0x7f08400ff1e0 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- 172.21.15.107:0/964332476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 msgr2=0x7f08400ff5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:52.605 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 --2- 172.21.15.107:0/964332476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f083c009840 tx=0x7f083c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- 172.21.15.107:0/964332476 shutdown_connections 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 --2- 172.21.15.107:0/964332476 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f08400ff5f0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- 172.21.15.107:0/964332476 >> 172.21.15.107:0/964332476 conn(0x7f08400faa00 msgr2=0x7f08400fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- 172.21.15.107:0/964332476 shutdown_connections 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- 172.21.15.107:0/964332476 wait complete. 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 Processor -- start 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.717+0000 7f0847534640 1 -- start start 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0847534640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0847534640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08401115b0 con 0x7f08400ff1e0 2024-09-10T18:48:52.606 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f08452d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:52.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f08452d0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36056/0 (socket says 172.21.15.107:36056) 2024-09-10T18:48:52.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f08452d0640 1 -- 172.21.15.107:0/1449682303 learned_addr learned my addr 172.21.15.107:0/1449682303 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:52.607 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f08452d0640 1 -- 172.21.15.107:0/1449682303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f083c009450 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f08452d0640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f083c006a70 tx=0x7f083c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f083c031e20 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f083c03b860 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f083c03a3a0 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.718+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08401117b0 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.719+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0840111c20 con 0x7f08400ff1e0 2024-09-10T18:48:52.608 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.719+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f083c039070 con 0x7f08400ff1e0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.719+0000 7f0835ffb640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 0x7f082c03b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.719+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f083c06c660 con 0x7f08400ff1e0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.719+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f084005ed70 con 0x7f08400ff1e0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.720+0000 7f0844acf640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 0x7f082c03b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.720+0000 7f0844acf640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 0x7f082c03b910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0830006fd0 tx=0x7f0830006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.724+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f083c03f080 con 0x7f08400ff1e0 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.914+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f0840112340 con 0x7f082c039460 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.921+0000 7f0835ffb640 1 -- 172.21.15.107:0/1449682303 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f0840112340 con 0x7f082c039460 2024-09-10T18:48:52.609 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 msgr2=0x7f082c03b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 0x7f082c03b910 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f0830006fd0 tx=0x7f0830006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 msgr2=0x7f0840111070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f083c006a70 tx=0x7f083c02c040 comp rx=0 tx=0).stop 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 shutdown_connections 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f082c039460 0x7f082c03b910 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 --2- 172.21.15.107:0/1449682303 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f08400ff1e0 0x7f0840111070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:52.610 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 >> 172.21.15.107:0/1449682303 conn(0x7f08400faa00 msgr2=0x7f0840103ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:52.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 shutdown_connections 2024-09-10T18:48:52.611 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:51.923+0000 7f0847534640 1 -- 172.21.15.107:0/1449682303 wait complete. 2024-09-10T18:48:52.611 INFO:teuthology.orchestra.run.smithi107.stdout:Deploying unmanaged mgr service... 2024-09-10T18:48:52.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:52 smithi107 ceph-mon[29404]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:52.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:52 smithi107 ceph-mon[29404]: Saving service mon spec with placement count:5 2024-09-10T18:48:52.979 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:52 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:54.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-10T18:48:54.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.328+0000 7f6d6f0a7640 1 Processor -- start 2024-09-10T18:48:54.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6f0a7640 1 -- start start 2024-09-10T18:48:54.451 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6f0a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6f0a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d68100ca0 con 0x7f6d68100350 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6ce43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6ce43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36070/0 (socket says 172.21.15.107:36070) 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6ce43640 1 -- 172.21.15.107:0/1508463317 learned_addr learned my addr 172.21.15.107:0/1508463317 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.329+0000 7f6d6ce43640 1 -- 172.21.15.107:0/1508463317 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d68100de0 con 0x7f6d68100350 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.330+0000 7f6d6ce43640 1 --2- 172.21.15.107:0/1508463317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009840 tx=0x7f6d5c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12a181c288165e7c server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.330+0000 7f6d677fe640 1 -- 172.21.15.107:0/1508463317 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c02add0 con 0x7f6d68100350 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.330+0000 7f6d677fe640 1 -- 172.21.15.107:0/1508463317 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f6d5c032860 con 0x7f6d68100350 2024-09-10T18:48:54.452 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.330+0000 7f6d677fe640 1 -- 172.21.15.107:0/1508463317 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c031630 con 0x7f6d68100350 2024-09-10T18:48:54.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/1508463317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 msgr2=0x7f6d68100760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:54.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/1508463317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c009840 tx=0x7f6d5c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:54.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/1508463317 shutdown_connections 2024-09-10T18:48:54.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/1508463317 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d68100760 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:54.453 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/1508463317 >> 172.21.15.107:0/1508463317 conn(0x7f6d680fbb50 msgr2=0x7f6d680fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/1508463317 shutdown_connections 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/1508463317 wait complete. 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 Processor -- start 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.331+0000 7f6d6f0a7640 1 -- start start 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6f0a7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6f0a7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d6810f3e0 con 0x7f6d68100350 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6ce43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6ce43640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36076/0 (socket says 172.21.15.107:36076) 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6ce43640 1 -- 172.21.15.107:0/309621453 learned_addr learned my addr 172.21.15.107:0/309621453 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6ce43640 1 -- 172.21.15.107:0/309621453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5c009450 con 0x7f6d68100350 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6ce43640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c031ad0 tx=0x7f6d5c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c03b440 con 0x7f6d68100350 2024-09-10T18:48:54.454 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 937+0+0 (secure 0 0 0) 0x7f6d5c03b9c0 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6d5c03a3a0 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d6810f5e0 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.332+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d6810fa50 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.333+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f6d5c039070 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.333+0000 7f6d657fa640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 0x7f6d5803b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.333+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f6d5c06c720 con 0x7f6d68100350 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.333+0000 7f6d67fff640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 0x7f6d5803b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:54.455 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.333+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d6805ed70 con 0x7f6d68100350 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.334+0000 7f6d67fff640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 0x7f6d5803b910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006fd0 tx=0x7f6d54006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.338+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6d5c03f080 con 0x7f6d68100350 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.504+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) v1 -- 0x7f6d681021a0 con 0x7f6d58039460 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.508+0000 7f6d657fa640 1 -- 172.21.15.107:0/309621453 <== mgr.14116 v2:172.21.15.107:6800/2559986261 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f6d681021a0 con 0x7f6d58039460 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 msgr2=0x7f6d5803b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 0x7f6d5803b910 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f6d54006fd0 tx=0x7f6d54006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 msgr2=0x7f6d6810eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f6d5c031ad0 tx=0x7f6d5c02c040 comp rx=0 tx=0).stop 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 shutdown_connections 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f6d58039460 0x7f6d5803b910 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 --2- 172.21.15.107:0/309621453 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6d68100350 0x7f6d6810eea0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 >> 172.21.15.107:0/309621453 conn(0x7f6d680fbb50 msgr2=0x7f6d680fc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:54.456 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 shutdown_connections 2024-09-10T18:48:54.457 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:53.510+0000 7f6d6f0a7640 1 -- 172.21.15.107:0/309621453 wait complete. 2024-09-10T18:48:54.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:54 smithi107 ceph-mon[29404]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:48:54.700 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:54 smithi107 ceph-mon[29404]: Saving service mgr spec with placement count:2 2024-09-10T18:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:54 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:54 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:54.701 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:54 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:55.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa932cb6640 1 Processor -- start 2024-09-10T18:48:55.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa932cb6640 1 -- start start 2024-09-10T18:48:55.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa932cb6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:55.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa932cb6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c0ffb60 con 0x7fa92c0ff210 2024-09-10T18:48:55.726 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa930a52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa930a52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40038/0 (socket says 172.21.15.107:40038) 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa930a52640 1 -- 172.21.15.107:0/993033526 learned_addr learned my addr 172.21.15.107:0/993033526 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.874+0000 7fa930a52640 1 -- 172.21.15.107:0/993033526 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa92c0ffca0 con 0x7fa92c0ff210 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.875+0000 7fa930a52640 1 --2- 172.21.15.107:0/993033526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa920009840 tx=0x7fa9200091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c88b16c724f96d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.875+0000 7fa92b7fe640 1 -- 172.21.15.107:0/993033526 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa92002add0 con 0x7fa92c0ff210 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.875+0000 7fa92b7fe640 1 -- 172.21.15.107:0/993033526 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa920032880 con 0x7fa92c0ff210 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.875+0000 7fa92b7fe640 1 -- 172.21.15.107:0/993033526 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa920031630 con 0x7fa92c0ff210 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- 172.21.15.107:0/993033526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 msgr2=0x7fa92c0ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 --2- 172.21.15.107:0/993033526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa920009840 tx=0x7fa9200091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:55.727 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- 172.21.15.107:0/993033526 shutdown_connections 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 --2- 172.21.15.107:0/993033526 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c0ff620 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- 172.21.15.107:0/993033526 >> 172.21.15.107:0/993033526 conn(0x7fa92c0faa10 msgr2=0x7fa92c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- 172.21.15.107:0/993033526 shutdown_connections 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- 172.21.15.107:0/993033526 wait complete. 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 Processor -- start 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.876+0000 7fa932cb6640 1 -- start start 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa932cb6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa932cb6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa92c111bf0 con 0x7fa92c0ff210 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa930a52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa930a52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40044/0 (socket says 172.21.15.107:40044) 2024-09-10T18:48:55.728 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa930a52640 1 -- 172.21.15.107:0/1305873178 learned_addr learned my addr 172.21.15.107:0/1305873178 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa930a52640 1 -- 172.21.15.107:0/1305873178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa920009450 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa930a52640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa920006a70 tx=0x7fa920008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa920031da0 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa920031f00 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa92c111d30 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa92003b5c0 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.877+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa92c112140 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.878+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7fa920039070 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.878+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa92c05ed70 con 0x7fa92c0ff210 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.878+0000 7fa9297fa640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 0x7fa91c03b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:55.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.879+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7fa92006d300 con 0x7fa92c0ff210 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.879+0000 7fa92bfff640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 0x7fa91c03b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.879+0000 7fa92bfff640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 0x7fa91c03b960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa918006fd0 tx=0x7fa918006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:54.883+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa92004eb30 con 0x7fa92c0ff210 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.076+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) v1 -- 0x7fa92c1124e0 con 0x7fa92c0ff210 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.079+0000 7fa9297fa640 1 -- 172.21.15.107:0/1305873178 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) v1 ==== 142+0+0 (secure 0 0 0) 0x7fa920037030 con 0x7fa92c0ff210 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 msgr2=0x7fa91c03b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:55.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 0x7fa91c03b960 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fa918006fd0 tx=0x7fa918006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 msgr2=0x7fa92c074370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa920006a70 tx=0x7fa920008040 comp rx=0 tx=0).stop 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 shutdown_connections 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7fa91c0394b0 0x7fa91c03b960 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 --2- 172.21.15.107:0/1305873178 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa92c0ff210 0x7fa92c074370 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.081+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 >> 172.21.15.107:0/1305873178 conn(0x7fa92c0faa10 msgr2=0x7fa92c0fb400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.082+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 shutdown_connections 2024-09-10T18:48:55.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:55.082+0000 7fa932cb6640 1 -- 172.21.15.107:0/1305873178 wait complete. 2024-09-10T18:48:56.438 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1305873178' entity='client.admin' 2024-09-10T18:48:57.358 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.522+0000 7f829df02640 1 Processor -- start 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.522+0000 7f829df02640 1 -- start start 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.522+0000 7f829df02640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.522+0000 7f829df02640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82981034a0 con 0x7f8298074480 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.523+0000 7f82977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.523+0000 7f82977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40050/0 (socket says 172.21.15.107:40050) 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.523+0000 7f82977fe640 1 -- 172.21.15.107:0/519929060 learned_addr learned my addr 172.21.15.107:0/519929060 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.523+0000 7f82977fe640 1 -- 172.21.15.107:0/519929060 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82981035e0 con 0x7f8298074480 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f82977fe640 1 --2- 172.21.15.107:0/519929060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8288009840 tx=0x7f82880091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f187867a2b07806 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f82967fc640 1 -- 172.21.15.107:0/519929060 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f828802add0 con 0x7f8298074480 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f82967fc640 1 -- 172.21.15.107:0/519929060 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8288032880 con 0x7f8298074480 2024-09-10T18:48:57.359 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f82967fc640 1 -- 172.21.15.107:0/519929060 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8288031630 con 0x7f8298074480 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 -- 172.21.15.107:0/519929060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 msgr2=0x7f8298102f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 --2- 172.21.15.107:0/519929060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8288009840 tx=0x7f82880091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 -- 172.21.15.107:0/519929060 shutdown_connections 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 --2- 172.21.15.107:0/519929060 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f8298102f60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 -- 172.21.15.107:0/519929060 >> 172.21.15.107:0/519929060 conn(0x7f82980faa30 msgr2=0x7f82980fce40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 -- 172.21.15.107:0/519929060 shutdown_connections 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.524+0000 7f829df02640 1 -- 172.21.15.107:0/519929060 wait complete. 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.525+0000 7f829df02640 1 Processor -- start 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.525+0000 7f829df02640 1 -- start start 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.525+0000 7f829df02640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.525+0000 7f829df02640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f829810f470 con 0x7f8298074480 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f82977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:57.360 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f82977fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40064/0 (socket says 172.21.15.107:40064) 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f82977fe640 1 -- 172.21.15.107:0/690171735 learned_addr learned my addr 172.21.15.107:0/690171735 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f82977fe640 1 -- 172.21.15.107:0/690171735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8288009450 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f82977fe640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8288006a70 tx=0x7f828802c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8288031cc0 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8288031e20 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f829810f670 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f828803bdc0 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.526+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f829810fae0 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.527+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f8288039070 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.527+0000 7f827ffff640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 0x7f828003b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.527+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f829804b720 con 0x7f8298074480 2024-09-10T18:48:57.361 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.527+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f828806c5e0 con 0x7f8298074480 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.527+0000 7f8296ffd640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 0x7f828003b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.528+0000 7f8296ffd640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 0x7f828003b910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f828c00ad90 tx=0x7f828c009410 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.532+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8288037080 con 0x7f8298074480 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.687+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) v1 -- 0x7f829810fae0 con 0x7f8298074480 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.690+0000 7f827ffff640 1 -- 172.21.15.107:0/690171735 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) v1 ==== 130+0+0 (secure 0 0 0) 0x7f828803f0c0 con 0x7f8298074480 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 msgr2=0x7f828003b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 0x7f828003b910 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f828c00ad90 tx=0x7f828c009410 comp rx=0 tx=0).stop 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 msgr2=0x7f829810ef30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8288006a70 tx=0x7f828802c040 comp rx=0 tx=0).stop 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 shutdown_connections 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8280039460 0x7f828003b910 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:57.362 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 --2- 172.21.15.107:0/690171735 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8298074480 0x7f829810ef30 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 >> 172.21.15.107:0/690171735 conn(0x7f82980faa30 msgr2=0x7f82980fb4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 shutdown_connections 2024-09-10T18:48:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:56.692+0000 7f829df02640 1 -- 172.21.15.107:0/690171735 wait complete. 2024-09-10T18:48:57.363 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling the dashboard module... 2024-09-10T18:48:57.770 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/690171735' entity='client.admin' 2024-09-10T18:48:57.771 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:57 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:58.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3816218164' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-09-10T18:48:58.897 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-mon[29404]: from='mgr.14116 172.21.15.107:0/2360565413' entity='mgr.y' 2024-09-10T18:48:58.897 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setuser ceph since I am not root 2024-09-10T18:48:58.897 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setgroup ceph since I am not root 2024-09-10T18:48:58.897 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:58.840+0000 7f490a04eec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:48:58.898 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:58.895+0000 7f490a04eec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:48:59.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:58.960+0000 7f490a04eec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:48:59.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:59.086+0000 7f490a04eec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:48:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824eb9d640 1 Processor -- start 2024-09-10T18:48:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824eb9d640 1 -- start start 2024-09-10T18:48:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824eb9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824eb9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8248076070 con 0x7f8248075c60 2024-09-10T18:48:59.376 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824db9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824db9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40076/0 (socket says 172.21.15.107:40076) 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.770+0000 7f824db9b640 1 -- 172.21.15.107:0/632463382 learned_addr learned my addr 172.21.15.107:0/632463382 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.771+0000 7f824db9b640 1 -- 172.21.15.107:0/632463382 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8248074770 con 0x7f8248075c60 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.771+0000 7f824db9b640 1 --2- 172.21.15.107:0/632463382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8244009840 tx=0x7f82440091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f81cde90fe40598d server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.771+0000 7f824cb99640 1 -- 172.21.15.107:0/632463382 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f824402add0 con 0x7f8248075c60 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.771+0000 7f824cb99640 1 -- 172.21.15.107:0/632463382 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8244032880 con 0x7f8248075c60 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824cb99640 1 -- 172.21.15.107:0/632463382 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8244031630 con 0x7f8248075c60 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 -- 172.21.15.107:0/632463382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 msgr2=0x7f8248074230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 --2- 172.21.15.107:0/632463382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8244009840 tx=0x7f82440091d0 comp rx=0 tx=0).stop 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 -- 172.21.15.107:0/632463382 shutdown_connections 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 --2- 172.21.15.107:0/632463382 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248074230 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 -- 172.21.15.107:0/632463382 >> 172.21.15.107:0/632463382 conn(0x7f82480fb990 msgr2=0x7f82480fddc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:59.377 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 -- 172.21.15.107:0/632463382 shutdown_connections 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.772+0000 7f824eb9d640 1 -- 172.21.15.107:0/632463382 wait complete. 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824eb9d640 1 Processor -- start 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824eb9d640 1 -- start start 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824eb9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824eb9d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82481114d0 con 0x7f8248075c60 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824db9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824db9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40086/0 (socket says 172.21.15.107:40086) 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824db9b640 1 -- 172.21.15.107:0/3816218164 learned_addr learned my addr 172.21.15.107:0/3816218164 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824db9b640 1 -- 172.21.15.107:0/3816218164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8244009450 con 0x7f8248075c60 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824db9b640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8244006a70 tx=0x7f824402c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8244031cc0 con 0x7f8248075c60 2024-09-10T18:48:59.378 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82481116d0 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.773+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8244031e20 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.774+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f824403bdc0 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.774+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8248111b40 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.774+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 8) v1 ==== 45492+0+0 (secure 0 0 0) 0x7f8244039070 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.775+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f824804b720 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.775+0000 7f823e7fc640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 0x7f823403b910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.775+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f824406c5d0 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.775+0000 7f824d39a640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 0x7f823403b910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.775+0000 7f824d39a640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 0x7f823403b910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8238006fd0 tx=0x7f8238006db0 comp rx=0 tx=0).ready entity=mgr.14116 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:57.779+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8244039320 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.008+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) v1 -- 0x7f8248111b40 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.696+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f82440433b0 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.697+0000 7f823e7fc640 1 -- 172.21.15.107:0/3816218164 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) v1 ==== 88+0+0 (secure 0 0 0) 0x7f8244038280 con 0x7f8248075c60 2024-09-10T18:48:59.379 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 msgr2=0x7f823403b910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 0x7f823403b910 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8238006fd0 tx=0x7f8238006db0 comp rx=0 tx=0).stop 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 msgr2=0x7f8248110f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8244006a70 tx=0x7f824402c040 comp rx=0 tx=0).stop 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 shutdown_connections 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8234039460 0x7f823403b910 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 --2- 172.21.15.107:0/3816218164 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8248075c60 0x7f8248110f90 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 >> 172.21.15.107:0/3816218164 conn(0x7f82480fb990 msgr2=0x7f82480fc490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 shutdown_connections 2024-09-10T18:48:59.380 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:58.699+0000 7f824eb9d640 1 -- 172.21.15.107:0/3816218164 wait complete. 2024-09-10T18:48:59.477 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:59.221+0000 7f490a04eec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:48:59.729 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3816218164' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-10T18:48:59.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-mon[29404]: mgrmap e9: y(active, since 18s) 2024-09-10T18:48:59.730 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:59.482+0000 7f490a04eec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:48:59.730 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:59.542+0000 7f490a04eec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:48:59.730 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:48:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:48:59.711+0000 7f490a04eec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:49:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2297807025' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-09-10T18:49:00.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:00.572+0000 7f490a04eec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:49:00.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:00.638+0000 7f490a04eec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:49:01.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:49:01.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-10T18:49:01.245 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "active_name": "y", 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.806+0000 7f8030da5640 1 Processor -- start 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.806+0000 7f8030da5640 1 -- start start 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.806+0000 7f8030da5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.806+0000 7f8030da5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c1010e0 con 0x7f802c100790 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.807+0000 7f802a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.807+0000 7f802a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40112/0 (socket says 172.21.15.107:40112) 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.807+0000 7f802a59c640 1 -- 172.21.15.107:0/3250322462 learned_addr learned my addr 172.21.15.107:0/3250322462 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.807+0000 7f802a59c640 1 -- 172.21.15.107:0/3250322462 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f802c101220 con 0x7f802c100790 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f802a59c640 1 --2- 172.21.15.107:0/3250322462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f801c009840 tx=0x7f801c0091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d531fa68b3998887 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:01.246 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f802959a640 1 -- 172.21.15.107:0/3250322462 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f801c02add0 con 0x7f802c100790 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f802959a640 1 -- 172.21.15.107:0/3250322462 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f801c032880 con 0x7f802c100790 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f802959a640 1 -- 172.21.15.107:0/3250322462 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f801c031630 con 0x7f802c100790 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f8030da5640 1 -- 172.21.15.107:0/3250322462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 msgr2=0x7f802c100ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.808+0000 7f8030da5640 1 --2- 172.21.15.107:0/3250322462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f801c009840 tx=0x7f801c0091d0 comp rx=0 tx=0).stop 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- 172.21.15.107:0/3250322462 shutdown_connections 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 --2- 172.21.15.107:0/3250322462 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c100ba0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- 172.21.15.107:0/3250322462 >> 172.21.15.107:0/3250322462 conn(0x7f802c0fa180 msgr2=0x7f802c0fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- 172.21.15.107:0/3250322462 shutdown_connections 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- 172.21.15.107:0/3250322462 wait complete. 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 Processor -- start 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- start start 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:01.247 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f8030da5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f802c111be0 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f802a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f802a59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40114/0 (socket says 172.21.15.107:40114) 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.809+0000 7f802a59c640 1 -- 172.21.15.107:0/2297807025 learned_addr learned my addr 172.21.15.107:0/2297807025 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f802a59c640 1 -- 172.21.15.107:0/2297807025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f801c009450 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f802a59c640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f801c006a70 tx=0x7f801c02c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f801c031e20 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f801c03b880 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f802c111d20 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f801c03a3a0 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.810+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f802c112190 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f801c039070 con 0x7f802c100790 2024-09-10T18:49:01.248 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f802c04b720 con 0x7f802c100790 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f801affd640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 0x7f801403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f801c06c8b0 con 0x7f802c100790 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f8029d9b640 1 -- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 msgr2=0x7f801403b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.811+0000 7f8029d9b640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 0x7f801403b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:48:59.815+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f801c03bb60 con 0x7f802c100790 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.012+0000 7f8029d9b640 1 -- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 msgr2=0x7f801403b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.012+0000 7f8029d9b640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 0x7f801403b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.027+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) v1 -- 0x7f802c1124c0 con 0x7f802c100790 2024-09-10T18:49:01.249 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.027+0000 7f801affd640 1 -- 172.21.15.107:0/2297807025 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) v1 ==== 56+0+88 (secure 0 0 0) 0x7f801c03f070 con 0x7f802c100790 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 msgr2=0x7f801403b9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 0x7f801403b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 msgr2=0x7f802c0fef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f801c006a70 tx=0x7f801c02c040 comp rx=0 tx=0).stop 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 shutdown_connections 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f8014039520 0x7f801403b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 --2- 172.21.15.107:0/2297807025 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f802c100790 0x7f802c0fef10 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 >> 172.21.15.107:0/2297807025 conn(0x7f802c0fa180 msgr2=0x7f802c0fab70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 shutdown_connections 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:00.030+0000 7f8030da5640 1 -- 172.21.15.107:0/2297807025 wait complete. 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for the mgr to restart... 2024-09-10T18:49:01.250 INFO:teuthology.orchestra.run.smithi107.stdout:Waiting for mgr epoch 9... 2024-09-10T18:49:01.600 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.356+0000 7f490a04eec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:49:01.600 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.475+0000 7f490a04eec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:49:01.854 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.598+0000 7f490a04eec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:49:01.854 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.660+0000 7f490a04eec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:49:01.854 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.786+0000 7f490a04eec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:49:01.854 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.852+0000 7f490a04eec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:49:02.108 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:01.910+0000 7f490a04eec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:49:02.108 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.043+0000 7f490a04eec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:49:02.108 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.106+0000 7f490a04eec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:49:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.171+0000 7f490a04eec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:49:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.251+0000 7f490a04eec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:49:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.308+0000 7f490a04eec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:49:02.367 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:02.365+0000 7f490a04eec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:03.108+0000 7f490a04eec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: Active manager daemon y restarted 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: Activating manager daemon y 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: osdmap e3: 0 total, 0 up, 0 in 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: mgrmap e10: y(active, starting, since 0.00320679s) 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:49:03.307 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: Manager daemon y is now available 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:49:03.308 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:49:04.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:04.523+0000 7f48babab640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:49:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:49:04.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Traceback (most recent call last): 2024-09-10T18:49:04.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:49:04.966 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._connections.run(self.expiration_interval) 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._run(expiration_interval) 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s = self.context.wrap_socket( 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: return self.sslsocket_class._create( 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self.do_handshake() 2024-09-10T18:49:04.967 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:49:04.968 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._sslobj.do_handshake() 2024-09-10T18:49:04.968 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:49:04.968 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:49:04 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout { 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout } 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.641+0000 7f9767e69640 1 Processor -- start 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9767e69640 1 -- start start 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9767e69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:04.980 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9767e69640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97600ffb20 con 0x7f97600ff1d0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9765c05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9765c05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40122/0 (socket says 172.21.15.107:40122) 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9765c05640 1 -- 172.21.15.107:0/3520438681 learned_addr learned my addr 172.21.15.107:0/3520438681 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.642+0000 7f9765c05640 1 -- 172.21.15.107:0/3520438681 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97600ffc60 con 0x7f97600ff1d0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.643+0000 7f9765c05640 1 --2- 172.21.15.107:0/3520438681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9750009840 tx=0x7f97500091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e9f2ad5ad5487 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.643+0000 7f9764c03640 1 -- 172.21.15.107:0/3520438681 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f975002add0 con 0x7f97600ff1d0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.643+0000 7f9764c03640 1 -- 172.21.15.107:0/3520438681 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f9750032880 con 0x7f97600ff1d0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.643+0000 7f9764c03640 1 -- 172.21.15.107:0/3520438681 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9750031630 con 0x7f97600ff1d0 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- 172.21.15.107:0/3520438681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 msgr2=0x7f97600ff5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 --2- 172.21.15.107:0/3520438681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f9750009840 tx=0x7f97500091d0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- 172.21.15.107:0/3520438681 shutdown_connections 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 --2- 172.21.15.107:0/3520438681 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f97600ff5e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.981 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- 172.21.15.107:0/3520438681 >> 172.21.15.107:0/3520438681 conn(0x7f97600fa9d0 msgr2=0x7f97600fcde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- 172.21.15.107:0/3520438681 shutdown_connections 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- 172.21.15.107:0/3520438681 wait complete. 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 Processor -- start 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.644+0000 7f9767e69640 1 -- start start 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9767e69640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9767e69640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f976010f460 con 0x7f97600ff1d0 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9765c05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9765c05640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:40130/0 (socket says 172.21.15.107:40130) 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9765c05640 1 -- 172.21.15.107:0/3201432044 learned_addr learned my addr 172.21.15.107:0/3201432044 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9765c05640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9750009450 con 0x7f97600ff1d0 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9765c05640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9750006a70 tx=0x7f9750008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f9750031da0 con 0x7f97600ff1d0 2024-09-10T18:49:04.982 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f9750031f00 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f976010f660 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f975003b5c0 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.645+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f976010fad0 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.646+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 9) v1 ==== 45505+0+0 (secure 0 0 0) 0x7f9750039070 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.646+0000 7f97567fc640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.646+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f974c03c0e0 con 0x7f974c039520 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.647+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 936+0+0 (secure 0 0 0) 0x7f975006d040 con 0x7f97600ff1d0 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.647+0000 7f9765404640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 msgr2=0x7f974c03b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.647+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.848+0000 7f9765404640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 msgr2=0x7f974c03b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:01.848+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:02.249+0000 7f9765404640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 msgr2=0x7f974c03b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:02.249+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:03.050+0000 7f9765404640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 msgr2=0x7f974c03b9d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/2559986261 2024-09-10T18:49:04.983 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:03.050+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:03.111+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mgrmap(e 10) v1 ==== 45272+0+0 (secure 0 0 0) 0x7f975006c1e0 con 0x7f97600ff1d0 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:03.111+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 msgr2=0x7f974c03b9d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:03.111+0000 7f97567fc640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.113+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 11) v1 ==== 45399+0+0 (secure 0 0 0) 0x7f975006ca40 con 0x7f97600ff1d0 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.113+0000 7f97567fc640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 0x7f974c03f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.113+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x7f974c03c0e0 con 0x7f974c03cfc0 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.114+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 0x7f974c03f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.114+0000 7f9765404640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 0x7f974c03f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f975c003940 tx=0x7f975c009350 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:04.984 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.115+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+6549 (secure 0 0 0) 0x7f974c03c0e0 con 0x7f974c03cfc0 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.119+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- command(tid 1: {"prefix": "mgr_status"}) v1 -- 0x7f976010fad0 con 0x7f974c03cfc0 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f97567fc640 1 -- 172.21.15.107:0/3201432044 <== mgr.14148 v2:172.21.15.107:6800/383804209 2 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+52 (secure 0 0 0) 0x7f976010fad0 con 0x7f974c03cfc0 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 msgr2=0x7f974c03f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 0x7f974c03f3a0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f975c003940 tx=0x7f975c009350 comp rx=0 tx=0).stop 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 msgr2=0x7f976010ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f9750006a70 tx=0x7f9750008040 comp rx=0 tx=0).stop 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 shutdown_connections 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f974c03cfc0 0x7f974c03f3a0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:6800/2559986261,v1:172.21.15.107:6801/2559986261] conn(0x7f974c039520 0x7f974c03b9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 --2- 172.21.15.107:0/3201432044 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f97600ff1d0 0x7f976010ef20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 >> 172.21.15.107:0/3201432044 conn(0x7f97600fa9d0 msgr2=0x7f97600fb800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 shutdown_connections 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:04.120+0000 7f9767e69640 1 -- 172.21.15.107:0/3201432044 wait complete. 2024-09-10T18:49:04.985 INFO:teuthology.orchestra.run.smithi107.stdout:mgr epoch 9 is available 2024-09-10T18:49:04.986 INFO:teuthology.orchestra.run.smithi107.stdout:Generating a dashboard self-signed certificate... 2024-09-10T18:49:05.224 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: mgrmap e11: y(active, since 1.00555s) 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: [10/Sep/2024:18:49:04] ENGINE Bus STARTING 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: [10/Sep/2024:18:49:04] ENGINE Serving on https://172.21.15.107:7150 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: [10/Sep/2024:18:49:04] ENGINE Bus STARTED 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: [10/Sep/2024:18:49:04] ENGINE Error in HTTPServer.serve 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:49:05.225 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:05.226 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:06.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-09-10T18:49:06.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.431+0000 7ffa2177f640 1 Processor -- start 2024-09-10T18:49:06.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.431+0000 7ffa2177f640 1 -- start start 2024-09-10T18:49:06.260 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa2177f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa2177f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c103480 con 0x7ffa1c0744d0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43270/0 (socket says 172.21.15.107:43270) 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa1affd640 1 -- 172.21.15.107:0/2267902636 learned_addr learned my addr 172.21.15.107:0/2267902636 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.432+0000 7ffa1affd640 1 -- 172.21.15.107:0/2267902636 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c1035c0 con 0x7ffa1c0744d0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.433+0000 7ffa1affd640 1 --2- 172.21.15.107:0/2267902636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009e50 tx=0x7ffa0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a803ae48734b2db server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.433+0000 7ffa19ffb640 1 -- 172.21.15.107:0/2267902636 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0c031400 con 0x7ffa1c0744d0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.433+0000 7ffa19ffb640 1 -- 172.21.15.107:0/2267902636 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7ffa0c02d050 con 0x7ffa1c0744d0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.433+0000 7ffa19ffb640 1 -- 172.21.15.107:0/2267902636 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0c007580 con 0x7ffa1c0744d0 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- 172.21.15.107:0/2267902636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 msgr2=0x7ffa1c102f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 --2- 172.21.15.107:0/2267902636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c009e50 tx=0x7ffa0c009840 comp rx=0 tx=0).stop 2024-09-10T18:49:06.261 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- 172.21.15.107:0/2267902636 shutdown_connections 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 --2- 172.21.15.107:0/2267902636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c102f40 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- 172.21.15.107:0/2267902636 >> 172.21.15.107:0/2267902636 conn(0x7ffa1c0faa10 msgr2=0x7ffa1c0fce20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- 172.21.15.107:0/2267902636 shutdown_connections 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- 172.21.15.107:0/2267902636 wait complete. 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 Processor -- start 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.434+0000 7ffa2177f640 1 -- start start 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa2177f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa2177f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c10f430 con 0x7ffa1c0744d0 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa1affd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43282/0 (socket says 172.21.15.107:43282) 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa1affd640 1 -- 172.21.15.107:0/962315841 learned_addr learned my addr 172.21.15.107:0/962315841 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa1affd640 1 -- 172.21.15.107:0/962315841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa0c009ae0 con 0x7ffa1c0744d0 2024-09-10T18:49:06.262 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa1affd640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007030 tx=0x7ffa0c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0c007ad0 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7ffa0c007c30 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1c10f630 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ffa0c044d70 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.435+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c10faa0 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.436+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 11) v1 ==== 45399+0+0 (secure 0 0 0) 0x7ffa0c038070 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.436+0000 7ffa137fe640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 0x7ffa0803b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.436+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ffa0c06cc80 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.436+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c05ed70 con 0x7ffa1c0744d0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.436+0000 7ffa1a7fc640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 0x7ffa0803b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.437+0000 7ffa1a7fc640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 0x7ffa0803b840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006fd0 tx=0x7ffa04006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:06.263 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.441+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ffa0c036070 con 0x7ffa1c0744d0 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.586+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 12) v1 ==== 45505+0+0 (secure 0 0 0) 0x7ffa0c03b460 con 0x7ffa1c0744d0 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.629+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa1c104400 con 0x7ffa08039390 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.689+0000 7ffa137fe640 1 -- 172.21.15.107:0/962315841 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7ffa1c104400 con 0x7ffa08039390 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 msgr2=0x7ffa0803b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 0x7ffa0803b840 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7ffa04006fd0 tx=0x7ffa04006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 msgr2=0x7ffa1c10eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c007030 tx=0x7ffa0c009840 comp rx=0 tx=0).stop 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 shutdown_connections 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ffa08039390 0x7ffa0803b840 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 --2- 172.21.15.107:0/962315841 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ffa1c0744d0 0x7ffa1c10eef0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:06.264 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.691+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 >> 172.21.15.107:0/962315841 conn(0x7ffa1c0faa10 msgr2=0x7ffa1c0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:06.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.692+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 shutdown_connections 2024-09-10T18:49:06.265 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:05.692+0000 7ffa2177f640 1 -- 172.21.15.107:0/962315841 wait complete. 2024-09-10T18:49:06.265 INFO:teuthology.orchestra.run.smithi107.stdout:Creating initial admin user... 2024-09-10T18:49:06.788 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:06.788 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:06 smithi107 ceph-mon[29404]: mgrmap e12: y(active, since 2s) 2024-09-10T18:49:06.788 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:06 smithi107 ceph-mon[29404]: from='client.14160 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:06.789 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:06.789 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:08 smithi107 ceph-mon[29404]: from='client.14162 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:08.572 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$Q8bb7q7iHAPASqP78AdulOMZQJ8czKD1lJ8OCctXuZ5VXoFx/V6Rm", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1725994147, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.701+0000 7fa408463640 1 Processor -- start 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.701+0000 7fa408463640 1 -- start start 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa408463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa408463640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa400101110 con 0x7fa4001007c0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa4061ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa4061ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43284/0 (socket says 172.21.15.107:43284) 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa4061ff640 1 -- 172.21.15.107:0/1949503299 learned_addr learned my addr 172.21.15.107:0/1949503299 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.702+0000 7fa4061ff640 1 -- 172.21.15.107:0/1949503299 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa400101250 con 0x7fa4001007c0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.703+0000 7fa4061ff640 1 --2- 172.21.15.107:0/1949503299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00baf0 tx=0x7fa3fc00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=164fb3723bddcf23 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.703+0000 7fa4051fd640 1 -- 172.21.15.107:0/1949503299 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3fc032640 con 0x7fa4001007c0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.703+0000 7fa4051fd640 1 -- 172.21.15.107:0/1949503299 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa3fc032be0 con 0x7fa4001007c0 2024-09-10T18:49:08.573 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.703+0000 7fa4051fd640 1 -- 172.21.15.107:0/1949503299 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3fc031a00 con 0x7fa4001007c0 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 -- 172.21.15.107:0/1949503299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 msgr2=0x7fa400100bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 --2- 172.21.15.107:0/1949503299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc00baf0 tx=0x7fa3fc00b460 comp rx=0 tx=0).stop 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 -- 172.21.15.107:0/1949503299 shutdown_connections 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 --2- 172.21.15.107:0/1949503299 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa400100bd0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 -- 172.21.15.107:0/1949503299 >> 172.21.15.107:0/1949503299 conn(0x7fa4000fa180 msgr2=0x7fa4000fc590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 -- 172.21.15.107:0/1949503299 shutdown_connections 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.704+0000 7fa408463640 1 -- 172.21.15.107:0/1949503299 wait complete. 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa408463640 1 Processor -- start 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa408463640 1 -- start start 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa408463640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa408463640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa40010d310 con 0x7fa4001007c0 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa4061ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:08.574 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa4061ff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43290/0 (socket says 172.21.15.107:43290) 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.705+0000 7fa4061ff640 1 -- 172.21.15.107:0/4171769512 learned_addr learned my addr 172.21.15.107:0/4171769512 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa4061ff640 1 -- 172.21.15.107:0/4171769512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc00b700 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa4061ff640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc003080 tx=0x7fa3fc00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3fc032950 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa3fc03b860 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa40010d510 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa3fc03a530 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.706+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa40010df60 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.707+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45505+0+0 (secure 0 0 0) 0x7fa3fc03a760 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.707+0000 7fa3f2ffd640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 0x7fa3ec03b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.707+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa40010d920 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.707+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa3fc06f640 con 0x7fa4001007c0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.707+0000 7fa4059fe640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 0x7fa3ec03b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:08.575 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.708+0000 7fa4059fe640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 0x7fa3ec03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.712+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa3fc036030 con 0x7fa4001007c0 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:06.889+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) v1 -- 0x7fa40005eb60 con 0x7fa3ec039520 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.119+0000 7fa3f2ffd640 1 -- 172.21.15.107:0/4171769512 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+252 (secure 0 0 0) 0x7fa40005eb60 con 0x7fa3ec039520 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 msgr2=0x7fa3ec03b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 0x7fa3ec03b9d0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4006fd0 tx=0x7fa3f4006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 msgr2=0x7fa40010cdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa3fc003080 tx=0x7fa3fc00b460 comp rx=0 tx=0).stop 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 shutdown_connections 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa3ec039520 0x7fa3ec03b9d0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 --2- 172.21.15.107:0/4171769512 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa4001007c0 0x7fa40010cdd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 >> 172.21.15.107:0/4171769512 conn(0x7fa4000fa180 msgr2=0x7fa400103b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 shutdown_connections 2024-09-10T18:49:08.576 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:07.121+0000 7fa408463640 1 -- 172.21.15.107:0/4171769512 wait complete. 2024-09-10T18:49:08.577 INFO:teuthology.orchestra.run.smithi107.stdout:Fetching dashboard port number... 2024-09-10T18:49:09.729 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stdout 8443 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.035+0000 7fc7ac408640 1 Processor -- start 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.035+0000 7fc7ac408640 1 -- start start 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7ac408640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7ac408640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a40760e0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7aa1a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7aa1a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43294/0 (socket says 172.21.15.107:43294) 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7aa1a4640 1 -- 172.21.15.107:0/3682311798 learned_addr learned my addr 172.21.15.107:0/3682311798 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.036+0000 7fc7aa1a4640 1 -- 172.21.15.107:0/3682311798 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4074790 con 0x7fc7a4075cd0 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.037+0000 7fc7aa1a4640 1 --2- 172.21.15.107:0/3682311798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc794009e50 tx=0x7fc794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe33ed480d89187 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.037+0000 7fc7a91a2640 1 -- 172.21.15.107:0/3682311798 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc794031400 con 0x7fc7a4075cd0 2024-09-10T18:49:09.730 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.037+0000 7fc7a91a2640 1 -- 172.21.15.107:0/3682311798 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc79402d050 con 0x7fc7a4075cd0 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.037+0000 7fc7a91a2640 1 -- 172.21.15.107:0/3682311798 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc794007580 con 0x7fc7a4075cd0 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- 172.21.15.107:0/3682311798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 msgr2=0x7fc7a4074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3682311798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc794009e50 tx=0x7fc794009840 comp rx=0 tx=0).stop 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- 172.21.15.107:0/3682311798 shutdown_connections 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3682311798 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4074250 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- 172.21.15.107:0/3682311798 >> 172.21.15.107:0/3682311798 conn(0x7fc7a40fbb20 msgr2=0x7fc7a40fdf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- 172.21.15.107:0/3682311798 shutdown_connections 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- 172.21.15.107:0/3682311798 wait complete. 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 Processor -- start 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- start start 2024-09-10T18:49:09.731 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7ac408640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a4111680 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.038+0000 7fc7aa1a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7aa1a4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43306/0 (socket says 172.21.15.107:43306) 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7aa1a4640 1 -- 172.21.15.107:0/3821723668 learned_addr learned my addr 172.21.15.107:0/3821723668 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7aa1a4640 1 -- 172.21.15.107:0/3821723668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc794009ae0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7aa1a4640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc794007030 tx=0x7fc794009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc794007ad0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc794007c30 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a4111880 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fc794044d70 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.039+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a4111cf0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.732 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.040+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 12) v1 ==== 45505+0+0 (secure 0 0 0) 0x7fc794038070 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.040+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7a404b720 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.040+0000 7fc7a2ffd640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 0x7fc78c03b960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.040+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc79406cde0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.040+0000 7fc7a99a3640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 0x7fc78c03b960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.041+0000 7fc7a99a3640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 0x7fc78c03b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc798006fd0 tx=0x7fc798006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.045+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc794044410 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.120+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fc79403b460 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.201+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) v1 -- 0x7fc7a4111cf0 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.202+0000 7fc7a2ffd640 1 -- 172.21.15.107:0/3821723668 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) v1 ==== 112+0+5 (secure 0 0 0) 0x7fc794036020 con 0x7fc7a4075cd0 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.203+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 msgr2=0x7fc78c03b960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.203+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 0x7fc78c03b960 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fc798006fd0 tx=0x7fc798006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:09.733 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.203+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 msgr2=0x7fc7a4111140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.203+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc794007030 tx=0x7fc794009840 comp rx=0 tx=0).stop 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 shutdown_connections 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc78c0394b0 0x7fc78c03b960 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 --2- 172.21.15.107:0/3821723668 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc7a4075cd0 0x7fc7a4111140 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 >> 172.21.15.107:0/3821723668 conn(0x7fc7a40fbb20 msgr2=0x7fc7a40fc600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 shutdown_connections 2024-09-10T18:49:09.734 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:09.204+0000 7fc7ac408640 1 -- 172.21.15.107:0/3821723668 wait complete. 2024-09-10T18:49:09.736 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-10T18:49:09.737 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout disabled 2024-09-10T18:49:09.743 INFO:teuthology.orchestra.run.smithi107.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-10T18:49:09.743 INFO:teuthology.orchestra.run.smithi107.stdout:systemctl: stdout inactive 2024-09-10T18:49:09.744 INFO:teuthology.orchestra.run.smithi107.stdout:firewalld.service is not enabled 2024-09-10T18:49:09.744 INFO:teuthology.orchestra.run.smithi107.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout:Ceph Dashboard is now available at: 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout: URL: https://smithi107.front.sepia.ceph.com:8443/ 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout: User: admin 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout: Password: x8cklpi23r 2024-09-10T18:49:09.746 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:09.747 INFO:teuthology.orchestra.run.smithi107.stdout:Saving cluster configuration to /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config directory 2024-09-10T18:49:09.747 INFO:teuthology.orchestra.run.smithi107.stdout:Enabling autotune for osd_memory_target 2024-09-10T18:49:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:10 smithi107 ceph-mon[29404]: mgrmap e13: y(active, since 6s) 2024-09-10T18:49:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3821723668' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-09-10T18:49:10.905 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.176+0000 7f1dd3f74640 1 Processor -- start 2024-09-10T18:49:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.176+0000 7f1dd3f74640 1 -- start start 2024-09-10T18:49:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.176+0000 7f1dd3f74640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.176+0000 7f1dd3f74640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc100d20 con 0x7f1dcc073c20 2024-09-10T18:49:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.177+0000 7f1dd1d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:10.910 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.177+0000 7f1dd1d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43312/0 (socket says 172.21.15.107:43312) 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.177+0000 7f1dd1d10640 1 -- 172.21.15.107:0/1366490918 learned_addr learned my addr 172.21.15.107:0/1366490918 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.177+0000 7f1dd1d10640 1 -- 172.21.15.107:0/1366490918 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dcc105870 con 0x7f1dcc073c20 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd1d10640 1 --2- 172.21.15.107:0/1366490918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009e50 tx=0x7f1dbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9d47e60d42cd9e server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd0d0e640 1 -- 172.21.15.107:0/1366490918 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc031400 con 0x7f1dcc073c20 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd0d0e640 1 -- 172.21.15.107:0/1366490918 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f1dbc02d050 con 0x7f1dcc073c20 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd0d0e640 1 -- 172.21.15.107:0/1366490918 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc007580 con 0x7f1dcc073c20 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd3f74640 1 -- 172.21.15.107:0/1366490918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 msgr2=0x7f1dcc105330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/1366490918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc009e50 tx=0x7f1dbc009840 comp rx=0 tx=0).stop 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd3f74640 1 -- 172.21.15.107:0/1366490918 shutdown_connections 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/1366490918 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc105330 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.178+0000 7f1dd3f74640 1 -- 172.21.15.107:0/1366490918 >> 172.21.15.107:0/1366490918 conn(0x7f1dcc0faa00 msgr2=0x7f1dcc0fce10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 -- 172.21.15.107:0/1366490918 shutdown_connections 2024-09-10T18:49:10.911 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 -- 172.21.15.107:0/1366490918 wait complete. 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 Processor -- start 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 -- start start 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd3f74640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dcc111600 con 0x7f1dcc073c20 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd1d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd1d10640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43318/0 (socket says 172.21.15.107:43318) 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.179+0000 7f1dd1d10640 1 -- 172.21.15.107:0/3947815396 learned_addr learned my addr 172.21.15.107:0/3947815396 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dd1d10640 1 -- 172.21.15.107:0/3947815396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dbc009ae0 con 0x7f1dcc073c20 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dd1d10640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007030 tx=0x7f1dbc009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc007ad0 con 0x7f1dcc073c20 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1dcc111800 con 0x7f1dcc073c20 2024-09-10T18:49:10.912 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f1dbc007c30 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f1dbc044d70 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.180+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1dcc111c70 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.181+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dcc04b720 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.181+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f1dbc038070 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.181+0000 7f1dca7fc640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 0x7f1db403ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.181+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f1dbc06cef0 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.181+0000 7f1dd150f640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 0x7f1db403ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.182+0000 7f1dd150f640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 0x7f1db403ba00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.185+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f1dbc044410 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.343+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) v1 -- 0x7f1dcc106470 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.344+0000 7f1dca7fc640 1 -- 172.21.15.107:0/3947815396 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) v1 ==== 127+0+0 (secure 0 0 0) 0x7f1dbc036070 con 0x7f1dcc073c20 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 msgr2=0x7f1db403ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:10.913 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 0x7f1db403ba00 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f1dc0006fd0 tx=0x7f1dc0006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 msgr2=0x7f1dcc1110c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1dbc007030 tx=0x7f1dbc009840 comp rx=0 tx=0).stop 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 shutdown_connections 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f1db4039550 0x7f1db403ba00 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 --2- 172.21.15.107:0/3947815396 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f1dcc073c20 0x7f1dcc1110c0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 >> 172.21.15.107:0/3947815396 conn(0x7f1dcc0faa00 msgr2=0x7f1dcc0fb470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 shutdown_connections 2024-09-10T18:49:10.914 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:10.346+0000 7f1dd3f74640 1 -- 172.21.15.107:0/3947815396 wait complete. 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c46b8640 1 Processor -- start 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c46b8640 1 -- start start 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c46b8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c46b8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc076970 con 0x7f25bc076560 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c2454640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c2454640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43320/0 (socket says 172.21.15.107:43320) 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.298+0000 7f25c2454640 1 -- 172.21.15.107:0/2637645890 learned_addr learned my addr 172.21.15.107:0/2637645890 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.299+0000 7f25c2454640 1 -- 172.21.15.107:0/2637645890 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25bc075020 con 0x7f25bc076560 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.299+0000 7f25c2454640 1 --2- 172.21.15.107:0/2637645890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009840 tx=0x7f25b80091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3987529ee0a2825 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.299+0000 7f25c1452640 1 -- 172.21.15.107:0/2637645890 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b802add0 con 0x7f25bc076560 2024-09-10T18:49:12.150 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.299+0000 7f25c1452640 1 -- 172.21.15.107:0/2637645890 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f25b8032880 con 0x7f25bc076560 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.299+0000 7f25c1452640 1 -- 172.21.15.107:0/2637645890 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b8031630 con 0x7f25bc076560 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 -- 172.21.15.107:0/2637645890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 msgr2=0x7f25bc074ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 --2- 172.21.15.107:0/2637645890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f25b8009840 tx=0x7f25b80091d0 comp rx=0 tx=0).stop 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 -- 172.21.15.107:0/2637645890 shutdown_connections 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 --2- 172.21.15.107:0/2637645890 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc074ae0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 -- 172.21.15.107:0/2637645890 >> 172.21.15.107:0/2637645890 conn(0x7f25bc0fbb50 msgr2=0x7f25bc0fdf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 -- 172.21.15.107:0/2637645890 shutdown_connections 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 -- 172.21.15.107:0/2637645890 wait complete. 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.300+0000 7f25c46b8640 1 Processor -- start 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c46b8640 1 -- start start 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c46b8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:12.151 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c46b8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25bc111620 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c2454640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c2454640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:43330/0 (socket says 172.21.15.107:43330) 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c2454640 1 -- 172.21.15.107:0/1630348125 learned_addr learned my addr 172.21.15.107:0/1630348125 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c2454640 1 -- 172.21.15.107:0/1630348125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25b8009450 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c2454640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25b80319c0 tx=0x7f25b8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b8031e50 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25bc111820 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f25b803d890 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f25b803b5a0 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.301+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25bc111c90 con 0x7f25bc076560 2024-09-10T18:49:12.152 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.302+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f25b8039070 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.302+0000 7f25b2ffd640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 0x7f25a803b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.302+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25bc04b720 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.302+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f25b806d2f0 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.303+0000 7f25c1c53640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 0x7f25a803b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.303+0000 7f25c1c53640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 0x7f25a803b9b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006fd0 tx=0x7f25ac006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.307+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f25b803dbe0 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.545+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) v1 -- 0x7f25bc111c90 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.548+0000 7f25b2ffd640 1 -- 172.21.15.107:0/1630348125 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) v1 ==== 153+0+0 (secure 0 0 0) 0x7f25b8041070 con 0x7f25bc076560 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 msgr2=0x7f25a803b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 0x7f25a803b9b0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f25ac006fd0 tx=0x7f25ac006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 msgr2=0x7f25bc1110e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:12.153 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f25b80319c0 tx=0x7f25b8008040 comp rx=0 tx=0).stop 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 shutdown_connections 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a8039500 0x7f25a803b9b0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 --2- 172.21.15.107:0/1630348125 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25bc076560 0x7f25bc1110e0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 >> 172.21.15.107:0/1630348125 conn(0x7f25bc0fbb50 msgr2=0x7f25bc0fc5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 shutdown_connections 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:/usr/bin/ceph: stderr 2024-09-10T18:49:11.550+0000 7f25c46b8640 1 -- 172.21.15.107:0/1630348125 wait complete. 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout:Or, if you are only running a single cluster on this host: 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-10T18:49:12.154 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: ceph telemetry on 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout:For more information see: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: https://docs.ceph.com/docs/master/mgr/telemetry/ 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:12.155 INFO:teuthology.orchestra.run.smithi107.stdout:Bootstrap complete. 2024-09-10T18:49:12.181 INFO:tasks.cephadm:Fetching config... 2024-09-10T18:49:12.181 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:12.181 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-10T18:49:12.197 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-10T18:49:12.198 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:12.198 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-10T18:49:12.252 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-10T18:49:12.252 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:12.252 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/keyring of=/dev/stdout 2024-09-10T18:49:12.320 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-10T18:49:12.320 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:12.320 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-10T18:49:12.373 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-10T18:49:12.373 DEBUG:teuthology.orchestra.run.smithi107:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC15SS9qEhoQcFBfLFwBnnu0rDpLFIJ4fqADixT0O7G3EWE3XFNQmXnb5DWANLlMzPzcOUoMuHOAzsglymZGYQMzl5Q/Z2j5sAM2h+s/SpUwFNIDCBk1z4lS3oMYFi8iMgvCZSOR2FicWv6IPIgjvFZrRKnV/oeDOgyylFewipR6UjQthp5hK+RtnnaNFO/AAG7jHmOCDZgstpPFA8jxYmBDC9SWHSSPoUpDu3REfEGKMRO3AdyboNxK30FcfkjH2A3a4+0KPGeZO7stGpLXAacIcZ/E8aRJFpbuhG7XEMzq3ZMokfTv6vYv2T5DxUtphq/pD5UxLR2YLOY+qRqK4L7ulA9iS+9WyuOb6LAtQWlYCk0MMo7YVpHJUPKqEbHvuIdluygNGo3BHZq4/bhRJyxVpWE58y2Awd3rfQeADnQ+stKGVjgxT+DkLEwK16IRR/qFRQZ5Y0qeSHomKkcXH9aCrutXd10WWS7SZJKWr+NDqk222lRehL9Gmu5i4Y2yek= ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:49:12.446 INFO:teuthology.orchestra.run.smithi107.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC15SS9qEhoQcFBfLFwBnnu0rDpLFIJ4fqADixT0O7G3EWE3XFNQmXnb5DWANLlMzPzcOUoMuHOAzsglymZGYQMzl5Q/Z2j5sAM2h+s/SpUwFNIDCBk1z4lS3oMYFi8iMgvCZSOR2FicWv6IPIgjvFZrRKnV/oeDOgyylFewipR6UjQthp5hK+RtnnaNFO/AAG7jHmOCDZgstpPFA8jxYmBDC9SWHSSPoUpDu3REfEGKMRO3AdyboNxK30FcfkjH2A3a4+0KPGeZO7stGpLXAacIcZ/E8aRJFpbuhG7XEMzq3ZMokfTv6vYv2T5DxUtphq/pD5UxLR2YLOY+qRqK4L7ulA9iS+9WyuOb6LAtQWlYCk0MMo7YVpHJUPKqEbHvuIdluygNGo3BHZq4/bhRJyxVpWE58y2Awd3rfQeADnQ+stKGVjgxT+DkLEwK16IRR/qFRQZ5Y0qeSHomKkcXH9aCrutXd10WWS7SZJKWr+NDqk222lRehL9Gmu5i4Y2yek= ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:12.457 DEBUG:teuthology.orchestra.run.smithi162:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC15SS9qEhoQcFBfLFwBnnu0rDpLFIJ4fqADixT0O7G3EWE3XFNQmXnb5DWANLlMzPzcOUoMuHOAzsglymZGYQMzl5Q/Z2j5sAM2h+s/SpUwFNIDCBk1z4lS3oMYFi8iMgvCZSOR2FicWv6IPIgjvFZrRKnV/oeDOgyylFewipR6UjQthp5hK+RtnnaNFO/AAG7jHmOCDZgstpPFA8jxYmBDC9SWHSSPoUpDu3REfEGKMRO3AdyboNxK30FcfkjH2A3a4+0KPGeZO7stGpLXAacIcZ/E8aRJFpbuhG7XEMzq3ZMokfTv6vYv2T5DxUtphq/pD5UxLR2YLOY+qRqK4L7ulA9iS+9WyuOb6LAtQWlYCk0MMo7YVpHJUPKqEbHvuIdluygNGo3BHZq4/bhRJyxVpWE58y2Awd3rfQeADnQ+stKGVjgxT+DkLEwK16IRR/qFRQZ5Y0qeSHomKkcXH9aCrutXd10WWS7SZJKWr+NDqk222lRehL9Gmu5i4Y2yek= ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-10T18:49:12.582 INFO:teuthology.orchestra.run.smithi162.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC15SS9qEhoQcFBfLFwBnnu0rDpLFIJ4fqADixT0O7G3EWE3XFNQmXnb5DWANLlMzPzcOUoMuHOAzsglymZGYQMzl5Q/Z2j5sAM2h+s/SpUwFNIDCBk1z4lS3oMYFi8iMgvCZSOR2FicWv6IPIgjvFZrRKnV/oeDOgyylFewipR6UjQthp5hK+RtnnaNFO/AAG7jHmOCDZgstpPFA8jxYmBDC9SWHSSPoUpDu3REfEGKMRO3AdyboNxK30FcfkjH2A3a4+0KPGeZO7stGpLXAacIcZ/E8aRJFpbuhG7XEMzq3ZMokfTv6vYv2T5DxUtphq/pD5UxLR2YLOY+qRqK4L7ulA9iS+9WyuOb6LAtQWlYCk0MMo7YVpHJUPKqEbHvuIdluygNGo3BHZq4/bhRJyxVpWE58y2Awd3rfQeADnQ+stKGVjgxT+DkLEwK16IRR/qFRQZ5Y0qeSHomKkcXH9aCrutXd10WWS7SZJKWr+NDqk222lRehL9Gmu5i4Y2yek= ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:12.596 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-10T18:49:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1630348125' entity='client.admin' 2024-09-10T18:49:12.843 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 -- 172.21.15.107:0/3833128868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 msgr2=0x7fec580feff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/3833128868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec580feff0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fec48004850 tx=0x7fec48009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 -- 172.21.15.107:0/3833128868 shutdown_connections 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/3833128868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec580feff0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 -- 172.21.15.107:0/3833128868 >> 172.21.15.107:0/3833128868 conn(0x7fec580f6ac0 msgr2=0x7fec580f8ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 -- 172.21.15.107:0/3833128868 shutdown_connections 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.524+0000 7fec5d7e8640 1 -- 172.21.15.107:0/3833128868 wait complete. 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec5d7e8640 1 Processor -- start 2024-09-10T18:49:15.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec5d7e8640 1 -- start start 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec5d7e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec5d7e8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec5810c1f0 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec56ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec56ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59162/0 (socket says 172.21.15.107:59162) 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec56ffd640 1 -- 172.21.15.107:0/18392011 learned_addr learned my addr 172.21.15.107:0/18392011 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.525+0000 7fec56ffd640 1 -- 172.21.15.107:0/18392011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec48005040 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec56ffd640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec480094b0 tx=0x7fec48009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec48038070 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec5810c3f0 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fec480098e0 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fec480335c0 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.526+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec5810cf80 con 0x7fec58065650 2024-09-10T18:49:15.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fec48030020 con 0x7fec58065650 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec3f7fe640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 0x7fec4003ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fec4803d9f0 con 0x7fec58065650 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec567fc640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 0x7fec4003ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec5810c8c0 con 0x7fec58065650 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.527+0000 7fec567fc640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 0x7fec4003ba00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006fd0 tx=0x7fec4c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:15.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.535+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fec5810c8c0 con 0x7fec58065650 2024-09-10T18:49:15.699 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.697+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) v1 -- 0x7fec5810c8c0 con 0x7fec58065650 2024-09-10T18:49:15.702 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.700+0000 7fec3f7fe640 1 -- 172.21.15.107:0/18392011 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) v1 ==== 125+0+0 (secure 0 0 0) 0x7fec5810c8c0 con 0x7fec58065650 2024-09-10T18:49:15.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 msgr2=0x7fec4003ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 0x7fec4003ba00 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fec4c006fd0 tx=0x7fec4c006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 msgr2=0x7fec5810bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fec480094b0 tx=0x7fec48009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 shutdown_connections 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fec40039550 0x7fec4003ba00 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 --2- 172.21.15.107:0/18392011 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fec58065650 0x7fec5810bcb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 >> 172.21.15.107:0/18392011 conn(0x7fec580f6ac0 msgr2=0x7fec580f7330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 shutdown_connections 2024-09-10T18:49:15.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:15.702+0000 7fec5d7e8640 1 -- 172.21.15.107:0/18392011 wait complete. 2024-09-10T18:49:16.738 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-10T18:49:16.739 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-10T18:49:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/18392011' entity='client.admin' 2024-09-10T18:49:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi107", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:16.974 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 -- 172.21.15.107:0/3365235882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 msgr2=0x7ff26c0fc5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:18.552 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 --2- 172.21.15.107:0/3365235882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c0fc5d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff260004850 tx=0x7ff260009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 -- 172.21.15.107:0/3365235882 shutdown_connections 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 --2- 172.21.15.107:0/3365235882 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c0fc5d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 -- 172.21.15.107:0/3365235882 >> 172.21.15.107:0/3365235882 conn(0x7ff26c0f7980 msgr2=0x7ff26c0f9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 -- 172.21.15.107:0/3365235882 shutdown_connections 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.548+0000 7ff2739e2640 1 -- 172.21.15.107:0/3365235882 wait complete. 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff2739e2640 1 Processor -- start 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff2739e2640 1 -- start start 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff2739e2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff2739e2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff26c10bfe0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff27177e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff27177e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59176/0 (socket says 172.21.15.107:59176) 2024-09-10T18:49:18.553 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff27177e640 1 -- 172.21.15.107:0/1270579241 learned_addr learned my addr 172.21.15.107:0/1270579241 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff27177e640 1 -- 172.21.15.107:0/1270579241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff260005040 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.549+0000 7ff27177e640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff260009040 tx=0x7ff260009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.550+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff260038070 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.550+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7ff2600098e0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.550+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff26c10c1e0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.550+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7ff260033420 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.554 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.550+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff26c10cd70 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7ff260030020 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff26c10c6b0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff25dffb640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 0x7ff25803ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7ff26006bea0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff270f7d640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 0x7ff25803ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:18.555 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.551+0000 7ff270f7d640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 0x7ff25803ba00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:18.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.556+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff26c10c6b0 con 0x7ff26c0fc1c0 2024-09-10T18:49:18.718 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.715+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) v1 -- 0x7ff26c10c6b0 con 0x7ff258039550 2024-09-10T18:49:18.721 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.719+0000 7ff25dffb640 1 -- 172.21.15.107:0/1270579241 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+0 (secure 0 0 0) 0x7ff26c10c6b0 con 0x7ff258039550 2024-09-10T18:49:18.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 msgr2=0x7ff25803ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:18.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 0x7ff25803ba00 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff268006fd0 tx=0x7ff268006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 msgr2=0x7ff26c10baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:18.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff260009040 tx=0x7ff260009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.723 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 shutdown_connections 2024-09-10T18:49:18.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7ff258039550 0x7ff25803ba00 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 --2- 172.21.15.107:0/1270579241 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff26c0fc1c0 0x7ff26c10baa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:18.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 >> 172.21.15.107:0/1270579241 conn(0x7ff26c0f7980 msgr2=0x7ff26c0f8630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:18.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 shutdown_connections 2024-09-10T18:49:18.724 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:18.721+0000 7ff2739e2640 1 -- 172.21.15.107:0/1270579241 wait complete. 2024-09-10T18:49:19.410 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi162 2024-09-10T18:49:19.410 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:49:19.410 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.conf 2024-09-10T18:49:19.430 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:49:19.430 DEBUG:teuthology.orchestra.run.smithi162:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:19.488 INFO:tasks.cephadm:Adding host smithi162 to orchestrator... 2024-09-10T18:49:19.488 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch host add smithi162 2024-09-10T18:49:19.738 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='client.14172 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:20.054 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:49:20.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:20.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:20.055 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:20.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:20 smithi107 ceph-mon[29404]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:20.939 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:20 smithi107 ceph-mon[29404]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:20.940 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:20 smithi107 ceph-mon[29404]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 -- 172.21.15.107:0/1861685730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 msgr2=0x7f80d81025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 --2- 172.21.15.107:0/1861685730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d81025b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004850 tx=0x7f80c8009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 -- 172.21.15.107:0/1861685730 shutdown_connections 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 --2- 172.21.15.107:0/1861685730 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d81025b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 -- 172.21.15.107:0/1861685730 >> 172.21.15.107:0/1861685730 conn(0x7f80d80f7db0 msgr2=0x7f80d80fa1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 -- 172.21.15.107:0/1861685730 shutdown_connections 2024-09-10T18:49:21.141 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.139+0000 7f80e031b640 1 -- 172.21.15.107:0/1861685730 wait complete. 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80e031b640 1 Processor -- start 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80e031b640 1 -- start start 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80e031b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80e031b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80d81120c0 con 0x7f80d81021e0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80de0b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80de0b7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:59200/0 (socket says 172.21.15.107:59200) 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80de0b7640 1 -- 172.21.15.107:0/3335704362 learned_addr learned my addr 172.21.15.107:0/3335704362 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80de0b7640 1 -- 172.21.15.107:0/3335704362 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80c8005040 con 0x7f80d81021e0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.140+0000 7f80de0b7640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004820 tx=0x7f80c8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80c8038070 con 0x7f80d81021e0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f80c8009790 con 0x7f80d81021e0 2024-09-10T18:49:21.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f80c80323d0 con 0x7f80d81021e0 2024-09-10T18:49:21.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80d81122c0 con 0x7f80d81021e0 2024-09-10T18:49:21.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80d8112710 con 0x7f80d81021e0 2024-09-10T18:49:21.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.141+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f80c8030020 con 0x7f80d81021e0 2024-09-10T18:49:21.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.142+0000 7f80ceffd640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 0x7f80c403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:21.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.142+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80d80fd950 con 0x7f80d81021e0 2024-09-10T18:49:21.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.142+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f80c806be00 con 0x7f80d81021e0 2024-09-10T18:49:21.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.142+0000 7f80dd8b6640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 0x7f80c403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:21.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.142+0000 7f80dd8b6640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 0x7f80c403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80d4006fd0 tx=0x7f80d4006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:21.147 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.146+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f80c803c5b0 con 0x7f80d81021e0 2024-09-10T18:49:21.307 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:21.306+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}) v1 -- 0x7f80d81057f0 con 0x7f80c40395c0 2024-09-10T18:49:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:21 smithi107 ceph-mon[29404]: from='client.14174 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi162", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:23.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:22 smithi107 ceph-mon[29404]: Deploying cephadm binary to smithi162 2024-09-10T18:49:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.101+0000 7f80ceffd640 1 -- 172.21.15.107:0/3335704362 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f80d81057f0 con 0x7f80c40395c0 2024-09-10T18:49:24.103 INFO:teuthology.orchestra.run.smithi107.stdout:Added host 'smithi162' with addr '172.21.15.162' 2024-09-10T18:49:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 msgr2=0x7f80c403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 0x7f80c403ba70 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f80d4006fd0 tx=0x7f80d4006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 msgr2=0x7f80d8111b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:24.105 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f80c8004820 tx=0x7f80c8009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 shutdown_connections 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f80c40395c0 0x7f80c403ba70 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 --2- 172.21.15.107:0/3335704362 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80d81021e0 0x7f80d8111b80 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 >> 172.21.15.107:0/3335704362 conn(0x7f80d80f7db0 msgr2=0x7f80d80f8d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 shutdown_connections 2024-09-10T18:49:24.106 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:24.104+0000 7f80e031b640 1 -- 172.21.15.107:0/3335704362 wait complete. 2024-09-10T18:49:24.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:24 smithi107 ceph-mon[29404]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:24.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:24.387 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:24.644 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch host ls --format=json 2024-09-10T18:49:24.878 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:25.668 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:25 smithi107 ceph-mon[29404]: Added host smithi162 2024-09-10T18:49:25.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:25 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:25.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:25 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:26.349 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.322+0000 7f0d62c38640 1 -- 172.21.15.107:0/22288258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 msgr2=0x7f0d5c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.322+0000 7f0d62c38640 1 --2- 172.21.15.107:0/22288258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c1014d0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004850 tx=0x7f0d58009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 -- 172.21.15.107:0/22288258 shutdown_connections 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 --2- 172.21.15.107:0/22288258 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c1014d0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 -- 172.21.15.107:0/22288258 >> 172.21.15.107:0/22288258 conn(0x7f0d5c0f6d10 msgr2=0x7f0d5c0f9120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 -- 172.21.15.107:0/22288258 shutdown_connections 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 -- 172.21.15.107:0/22288258 wait complete. 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 Processor -- start 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.323+0000 7f0d62c38640 1 -- start start 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d62c38640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d62c38640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d5c110f60 con 0x7f0d5c101100 2024-09-10T18:49:26.350 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d609d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d609d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35394/0 (socket says 172.21.15.107:35394) 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d609d4640 1 -- 172.21.15.107:0/3937677762 learned_addr learned my addr 172.21.15.107:0/3937677762 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d609d4640 1 -- 172.21.15.107:0/3937677762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d58005040 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d609d4640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004820 tx=0x7f0d58009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.324+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d58038070 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.325+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f0d58009790 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.325+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d5c111160 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.325+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f0d580323d0 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.325+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d5c111630 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.325+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f0d58030020 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.326+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d5c0fc870 con 0x7f0d5c101100 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.326+0000 7f0d517fa640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 0x7f0d4403ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:26.351 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.326+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f0d5806bc70 con 0x7f0d5c101100 2024-09-10T18:49:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.326+0000 7f0d53fff640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 0x7f0d4403ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.326+0000 7f0d53fff640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 0x7f0d4403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:26.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.331+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f0d580453a0 con 0x7f0d5c101100 2024-09-10T18:49:26.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.491+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0d5c111630 con 0x7f0d440395c0 2024-09-10T18:49:26.493 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.492+0000 7f0d517fa640 1 -- 172.21.15.107:0/3937677762 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+161 (secure 0 0 0) 0x7f0d5c111630 con 0x7f0d440395c0 2024-09-10T18:49:26.494 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:49:26.494 INFO:teuthology.orchestra.run.smithi107.stdout:[{"addr": "172.21.15.107", "hostname": "smithi107", "labels": [], "status": ""}, {"addr": "172.21.15.162", "hostname": "smithi162", "labels": [], "status": ""}] 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 msgr2=0x7f0d4403ba70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 0x7f0d4403ba70 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0d4c006fd0 tx=0x7f0d4c006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 msgr2=0x7f0d5c110a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0d58004820 tx=0x7f0d58009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 shutdown_connections 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f0d440395c0 0x7f0d4403ba70 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 --2- 172.21.15.107:0/3937677762 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f0d5c101100 0x7f0d5c110a20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 >> 172.21.15.107:0/3937677762 conn(0x7f0d5c0f6d10 msgr2=0x7f0d5c0f7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 shutdown_connections 2024-09-10T18:49:26.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:26.494+0000 7f0d62c38640 1 -- 172.21.15.107:0/3937677762 wait complete. 2024-09-10T18:49:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:26 smithi107 ceph-mon[29404]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:26 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:27.218 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-10T18:49:27.218 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd crush tunables default 2024-09-10T18:49:27.455 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:27 smithi107 ceph-mon[29404]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd/host:smithi162", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:28.590 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:29.049 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.047+0000 7f8d45d59640 1 -- 172.21.15.107:0/2813092853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 msgr2=0x7f8d40102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.047+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2813092853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40102620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f8d30004850 tx=0x7f8d30009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.047+0000 7f8d45d59640 1 -- 172.21.15.107:0/2813092853 shutdown_connections 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.047+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2813092853 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40102620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.047+0000 7f8d45d59640 1 -- 172.21.15.107:0/2813092853 >> 172.21.15.107:0/2813092853 conn(0x7f8d40071790 msgr2=0x7f8d40071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.048+0000 7f8d45d59640 1 -- 172.21.15.107:0/2813092853 shutdown_connections 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.048+0000 7f8d45d59640 1 -- 172.21.15.107:0/2813092853 wait complete. 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.048+0000 7f8d45d59640 1 Processor -- start 2024-09-10T18:49:29.050 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.048+0000 7f8d45d59640 1 -- start start 2024-09-10T18:49:29.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d45d59640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:29.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d45d59640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d40112070 con 0x7f8d40102250 2024-09-10T18:49:29.051 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d3f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d3f7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:35404/0 (socket says 172.21.15.107:35404) 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d3f7fe640 1 -- 172.21.15.107:0/2671365947 learned_addr learned my addr 172.21.15.107:0/2671365947 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d3f7fe640 1 -- 172.21.15.107:0/2671365947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d30005040 con 0x7f8d40102250 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d3f7fe640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009ac0 tx=0x7f8d30009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d30038070 con 0x7f8d40102250 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d40112270 con 0x7f8d40102250 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8d300096e0 con 0x7f8d40102250 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f8d30033420 con 0x7f8d40102250 2024-09-10T18:49:29.052 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.049+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d40112740 con 0x7f8d40102250 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.050+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f8d30030020 con 0x7f8d40102250 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.050+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d40064750 con 0x7f8d40102250 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.050+0000 7f8d27fff640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 0x7f8d2803ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.051+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f8d3006bac0 con 0x7f8d40102250 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.051+0000 7f8d3effd640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 0x7f8d2803ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:29.053 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.051+0000 7f8d3effd640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 0x7f8d2803ba00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006fd0 tx=0x7f8d34006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:29.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.055+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8d3002cb80 con 0x7f8d40102250 2024-09-10T18:49:29.223 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:29.221+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f8d40112740 con 0x7f8d40102250 2024-09-10T18:49:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:49:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2671365947' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-09-10T18:49:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.039+0000 7f8d27fff640 1 -- 172.21.15.107:0/2671365947 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) v1 ==== 124+0+0 (secure 0 0 0) 0x7f8d40112740 con 0x7f8d40102250 2024-09-10T18:49:30.040 INFO:teuthology.orchestra.run.smithi107.stderr:adjusted tunables profile to default 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 msgr2=0x7f8d2803ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 0x7f8d2803ba00 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8d34006fd0 tx=0x7f8d34006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 msgr2=0x7f8d40111b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f8d30009ac0 tx=0x7f8d30009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 shutdown_connections 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8d28039550 0x7f8d2803ba00 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 --2- 172.21.15.107:0/2671365947 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8d40102250 0x7f8d40111b30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 >> 172.21.15.107:0/2671365947 conn(0x7f8d40071790 msgr2=0x7f8d40101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:30.043 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 shutdown_connections 2024-09-10T18:49:30.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:30.041+0000 7f8d45d59640 1 -- 172.21.15.107:0/2671365947 wait complete. 2024-09-10T18:49:30.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:30 smithi107 ceph-mon[29404]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:49:30.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:30 smithi107 ceph-mon[29404]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:30.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2671365947' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-10T18:49:30.638 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:30 smithi107 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:30.667 INFO:tasks.cephadm:Adding mon.a on smithi107 2024-09-10T18:49:30.667 INFO:tasks.cephadm:Adding mon.c on smithi107 2024-09-10T18:49:30.667 INFO:tasks.cephadm:Adding mon.b on smithi162 2024-09-10T18:49:30.668 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply mon '3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi162:172.21.15.162=b' 2024-09-10T18:49:30.912 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:32.471 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3933682237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 msgr2=0x7fa8b41027d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3933682237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b41027d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4004850 tx=0x7fa8a4009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3933682237 shutdown_connections 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3933682237 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b41027d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3933682237 >> 172.21.15.162:0/3933682237 conn(0x7fa8b4071790 msgr2=0x7fa8b4071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3933682237 shutdown_connections 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3933682237 wait complete. 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 Processor -- start 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.468+0000 7fa8bbcb9640 1 -- start start 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8bbcb9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:32.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8bbcb9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8b4112220 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b9a55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b9a55640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.162:37486/0 (socket says 172.21.15.162:37486) 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b9a55640 1 -- 172.21.15.162:0/3794999226 learned_addr learned my addr 172.21.15.162:0/3794999226 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b9a55640 1 -- 172.21.15.162:0/3794999226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8a4005040 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b9a55640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4004820 tx=0x7fa8a4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.469+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8a4038070 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8b4112420 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa8a4009790 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7fa8a40323d0 con 0x7fa8b4102400 2024-09-10T18:49:32.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8b41128f0 con 0x7fa8b4102400 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8b40fdb70 con 0x7fa8b4102400 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.470+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fa8a4030020 con 0x7fa8b4102400 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.471+0000 7fa8b27fc640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 0x7fa89c03ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.471+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fa8a406bea0 con 0x7fa8b4102400 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.471+0000 7fa8b9254640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 0x7fa89c03ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:32.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.471+0000 7fa8b9254640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 0x7fa89c03ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:32.477 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.475+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa8a4033420 con 0x7fa8b4102400 2024-09-10T18:49:32.640 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.637+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi162:172.21.15.162=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fa8b41059b0 con 0x7fa89c039550 2024-09-10T18:49:32.646 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.644+0000 7fa8b27fc640 1 -- 172.21.15.162:0/3794999226 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7fa8b41059b0 con 0x7fa89c039550 2024-09-10T18:49:32.646 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mon update... 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 msgr2=0x7fa89c03ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 0x7fa89c03ba00 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa8a8006fd0 tx=0x7fa8a8006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 msgr2=0x7fa8b4111ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fa8a4004820 tx=0x7fa8a4009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 shutdown_connections 2024-09-10T18:49:32.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fa89c039550 0x7fa89c03ba00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 --2- 172.21.15.162:0/3794999226 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8b4102400 0x7fa8b4111ce0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:32.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 >> 172.21.15.162:0/3794999226 conn(0x7fa8b4071790 msgr2=0x7fa8b40f9af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:32.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 shutdown_connections 2024-09-10T18:49:32.649 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:32.646+0000 7fa8bbcb9640 1 -- 172.21.15.162:0/3794999226 wait complete. 2024-09-10T18:49:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:32 smithi107 ceph-mon[29404]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:33.227 DEBUG:teuthology.orchestra.run.smithi107:mon.c> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.c.service 2024-09-10T18:49:33.230 DEBUG:teuthology.orchestra.run.smithi162:mon.b> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.b.service 2024-09-10T18:49:33.232 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:49:33.232 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:49:33.965 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi162:172.21.15.162=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: Saving service mon spec with placement smithi107:172.21.15.107=a;smithi107:[v2:172.21.15.107:3301,v1:172.21.15.107:6790]=c;smithi162:172.21.15.162=b;count:3 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:33 smithi107 ceph-mon[29404]: Deploying daemon mon.b on smithi162 2024-09-10T18:49:34.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:34 smithi107 ceph-mon[29404]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:36.455 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:36 smithi162 systemd[1]: Starting Ceph mon.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:49:36.815 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 -- 172.21.15.162:0/2791197712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 msgr2=0x7f622c1014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 --2- 172.21.15.162:0/2791197712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c1014d0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f6228004850 tx=0x7f6228009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 -- 172.21.15.162:0/2791197712 shutdown_connections 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 --2- 172.21.15.162:0/2791197712 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c1014d0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 -- 172.21.15.162:0/2791197712 >> 172.21.15.162:0/2791197712 conn(0x7f622c0f6cd0 msgr2=0x7f622c0f90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 -- 172.21.15.162:0/2791197712 shutdown_connections 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.810+0000 7f62342f3640 1 -- 172.21.15.162:0/2791197712 wait complete. 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f62342f3640 1 Processor -- start 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f62342f3640 1 -- start start 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f62342f3640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:36.816 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f62342f3640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f622c10d020 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f623208f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f623208f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.162:47186/0 (socket says 172.21.15.162:47186) 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f623208f640 1 -- 172.21.15.162:0/2647778447 learned_addr learned my addr 172.21.15.162:0/2647778447 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.811+0000 7f623208f640 1 -- 172.21.15.162:0/2647778447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6228005040 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f623208f640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6228009040 tx=0x7f6228009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6228038070 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f6228009790 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f622c10d220 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 205+0+0 (secure 0 0 0) 0x7f6228033420 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.812+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f622c10d6f0 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.813+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f6228030020 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.813+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f622c10d930 con 0x7f622c101100 2024-09-10T18:49:36.817 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.813+0000 7f6222ffd640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 0x7f621803ba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:36.818 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.813+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f622806be90 con 0x7f622c101100 2024-09-10T18:49:36.818 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.813+0000 7f623188e640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 0x7f621803ba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:36.818 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.814+0000 7f623188e640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 0x7f621803ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006db0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:36.819 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:36.818+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6228033870 con 0x7f622c101100 2024-09-10T18:49:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:36 smithi107 ceph-mon[29404]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:37.040 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.038+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7f622c102db0 con 0x7f622c101100 2024-09-10T18:49:37.041 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.038+0000 7f6222ffd640 1 -- 172.21.15.162:0/2647778447 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) v1 ==== 95+0+735 (secure 0 0 0) 0x7f622802c110 con 0x7f622c101100 2024-09-10T18:49:37.041 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:49:37.041 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":1,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","modified":"2024-09-10T18:48:14.624206Z","created":"2024-09-10T18:48:14.624206Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-10T18:49:37.041 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 1 2024-09-10T18:49:37.042 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 msgr2=0x7f621803ba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 0x7f621803ba00 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f621c006fd0 tx=0x7f621c006db0 comp rx=0 tx=0).stop 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 msgr2=0x7f622c10cae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f6228009040 tx=0x7f6228009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 shutdown_connections 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f6218039550 0x7f621803ba00 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 --2- 172.21.15.162:0/2647778447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f622c101100 0x7f622c10cae0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.040+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 >> 172.21.15.162:0/2647778447 conn(0x7f622c0f6cd0 msgr2=0x7f622c0f7bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.041+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 shutdown_connections 2024-09-10T18:49:37.043 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:37.041+0000 7f62342f3640 1 -- 172.21.15.162:0/2647778447 wait complete. 2024-09-10T18:49:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:36 smithi162 podman[33177]: 2024-09-10 18:49:36.898084319 +0000 UTC m=+0.015761991 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:49:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 podman[33177]: 2024-09-10 18:49:37.040258434 +0000 UTC m=+0.157936100 container create fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 podman[33177]: 2024-09-10 18:49:37.460807419 +0000 UTC m=+0.578485094 container init fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 podman[33177]: 2024-09-10 18:49:37.465490175 +0000 UTC m=+0.583167843 container start fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: pidfile_write: ignore empty --pid-file 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: load: jerasure load: lrc 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:49:37.739 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: DB SUMMARY 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: DB Session ID: SO5J2134TB1ZKRBYBVC6 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: SST files in /var/lib/ceph/mon/ceph-b/store.db dir, Total Num: 0, files: 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-b/store.db: 000004.log size: 511 ; 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.env: 0x5571d9f95440 2024-09-10T18:49:37.740 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.fs: Posix File System 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.info_log: 0x5571db1beec0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.statistics: (nil) 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.use_fsync: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:49:37.741 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.db_log_dir: 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-b/store.db 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:49:37.742 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.write_buffer_manager: 0x5571db27c780 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.unordered_write: 0 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:49:37.743 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:49:37.744 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.row_cache: None 2024-09-10T18:49:37.744 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.wal_filter: None 2024-09-10T18:49:37.745 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:49:37.745 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:49:37.745 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:49:37.745 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:49:37.745 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:49:37.746 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_open_files: -1 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:49:37.747 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Compression algorithms supported: 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kZSTD supported: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:49:37.748 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.merge_operator: 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_filter: None 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5571db165a38) 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:49:37.749 INFO:journalctl@ceph.mon.b.smithi162.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_type: 0 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: data_block_index_type: 0 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_shortening: 1 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: hash_index_allow_collision: 1 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: checksum: 1 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: no_block_cache: 0 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache: 0x5571db19b1f0 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_options: 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: capacity : 536870912 2024-09-10T18:49:37.750 INFO:journalctl@ceph.mon.b.smithi162.stdout: num_shard_bits : 4 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: strict_capacity_limit : 0 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_cache_compressed: (nil) 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: persistent_cache: (nil) 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_size: 4096 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_size_deviation: 10 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_restart_interval: 16 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: index_block_restart_interval: 1 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: metadata_block_size: 4096 2024-09-10T18:49:37.751 INFO:journalctl@ceph.mon.b.smithi162.stdout: partition_filters: 0 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: use_delta_encoding: 1 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: whole_key_filtering: 1 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: verify_compression: 0 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: format_version: 4 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: enable_index_compression: 1 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout: block_align: 0 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:49:37.752 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression: NoCompression 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.num_levels: 7 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:49:37.753 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:49:37.754 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:49:37.755 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:49:37.756 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:49:37.757 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.ttl: 2592000 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:49:37.758 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-b/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994177489691, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994177490005, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1571, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725994177, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "7d6f87d3-0840-4015-85ff-83c1e58923d4", "db_session_id": "SO5J2134TB1ZKRBYBVC6"}} 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994177490326, "job": 1, "event": "recovery_finished"} 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-b/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5571db1ce700 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: DB pointer 0x5571db19e000 2024-09-10T18:49:37.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** DB Stats ** 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:37.760 INFO:journalctl@ceph.mon.b.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:37.761 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval compaction: 0.00 GB write, 0.52 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: L0 1/0 1.53 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.762 INFO:journalctl@ceph.mon.b.smithi162.stdout: Sum 1/0 1.53 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.1 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Cumulative compaction: 0.00 GB write, 0.51 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:37.763 INFO:journalctl@ceph.mon.b.smithi162.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: using public_addr v2:172.21.15.162:0/0 -> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: starting mon.b rank -1 at public addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] at bind addrs [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] mon_data /var/lib/ceph/mon/ceph-b fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(???) e0 preinit fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).mds e1 new map 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).mds e1 print_map 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: e1 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: legacy client fscid: -1 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: 2024-09-10T18:49:37.764 INFO:journalctl@ceph.mon.b.smithi162.stdout: No filesystems configured 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 ceph-mon[33228]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:49:37.765 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 bash[33177]: fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 2024-09-10T18:49:37.766 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:37 smithi162 systemd[1]: Started Ceph mon.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:49:39.088 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:49:39.089 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:49:39.357 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: mon.a calling monitor election 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: mon.b calling monitor election 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: fsmap 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: mgrmap e13: y(active, since 39s) 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: overall HEALTH_OK 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:42.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:42.959 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: mon.a calling monitor election 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: mon.b calling monitor election 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:42.960 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: fsmap 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: mgrmap e13: y(active, since 39s) 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: overall HEALTH_OK 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:42.961 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:43 smithi162 ceph-mon[33228]: Deploying daemon mon.c on smithi107 2024-09-10T18:49:43.885 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:43 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 -- 172.21.15.162:0/42713999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a40034f0 msgr2=0x7fc6a4005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 --2- 172.21.15.162:0/42713999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a40034f0 0x7fc6a4005970 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80fd400 tx=0x7fc6b400b040 comp rx=0 tx=0).stop 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 -- 172.21.15.162:0/42713999 shutdown_connections 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 --2- 172.21.15.162:0/42713999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6a40034f0 0x7fc6a4005970 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 -- 172.21.15.162:0/42713999 >> 172.21.15.162:0/42713999 conn(0x7fc6b8072030 msgr2=0x7fc6b8072430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 -- 172.21.15.162:0/42713999 shutdown_connections 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.883+0000 7fc6bf485640 1 -- 172.21.15.162:0/42713999 wait complete. 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 Processor -- start 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 -- start start 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 0x7fc6b80fbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:43.887 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80fc130 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bf485640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8118030 con 0x7fc6a40034f0 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48354/0 (socket says 172.21.15.162:48354) 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 -- 172.21.15.162:0/3146971259 learned_addr learned my addr 172.21.15.162:0/3146971259 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 -- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 msgr2=0x7fc6b80fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 -- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 msgr2=0x7fc6b80fb260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bca20640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 0x7fc6b80fbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.884+0000 7fc6bd221640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bca20640 1 -- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 msgr2=0x7fc6b80fb260 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bca20640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bca20640 1 -- 172.21.15.162:0/3146971259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b4005040 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.888 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bca20640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 0x7fc6b80fbbf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b580 tx=0x7fc6a80090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:43.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fc6a8011840 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b8118310 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc6a8011de0 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 309+0+0 (secure 0 0 0) 0x7fc6a8010cd0 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.889 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.885+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b8118830 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.886+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fc6a801a410 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.886+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b80fda50 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.886+0000 7fc6adffb640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 0x7fc69c03bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.887+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fc6a804c1d0 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.887+0000 7fc6bd221640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 0x7fc69c03bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:43.890 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.887+0000 7fc6bd221640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 0x7fc69c03bba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4007b40 tx=0x7fc6b400b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:43.892 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:43.891+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc6a80178a0 con 0x7fc6b80fb7a0 2024-09-10T18:49:43.959 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:43 smithi107 ceph-mon[29404]: Deploying daemon mon.c on smithi107 2024-09-10T18:49:43.959 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:43 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:44.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.115+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fc6b805ed70 con 0x7fc6b80fb7a0 2024-09-10T18:49:44.116 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.115+0000 7fc6adffb640 1 -- 172.21.15.162:0/3146971259 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) v1 ==== 95+0+1005 (secure 0 0 0) 0x7fc6a8015020 con 0x7fc6b80fb7a0 2024-09-10T18:49:44.117 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:49:44.117 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":2,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","modified":"2024-09-10T18:49:37.518527Z","created":"2024-09-10T18:48:14.624206Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:49:44.117 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 2 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.117+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 msgr2=0x7fc69c03bba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.117+0000 7fc6bf485640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 0x7fc69c03bba0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc6b4007b40 tx=0x7fc6b400b040 comp rx=0 tx=0).stop 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.117+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 msgr2=0x7fc6b80fbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.117+0000 7fc6bf485640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 0x7fc6b80fbbf0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc6a800b580 tx=0x7fc6a80090f0 comp rx=0 tx=0).stop 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 shutdown_connections 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc69c0396f0 0x7fc69c03bba0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc6b80fb7a0 0x7fc6b80fbbf0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 --2- 172.21.15.162:0/3146971259 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc6a40034f0 0x7fc6b80fb260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 >> 172.21.15.162:0/3146971259 conn(0x7fc6b8072030 msgr2=0x7fc6b80fdeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 shutdown_connections 2024-09-10T18:49:44.119 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:44.118+0000 7fc6bf485640 1 -- 172.21.15.162:0/3146971259 wait complete. 2024-09-10T18:49:44.726 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:44 smithi162 ceph-mon[33228]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:44.726 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/3146971259' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:44.915 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:44 smithi107 ceph-mon[29404]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:44.916 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/3146971259' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:45.606 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 podman[40317]: 2024-09-10 18:49:45.49113096 +0000 UTC m=+0.121008048 container create 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T18:49:45.752 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-10T18:49:45.752 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph mon dump -f json 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 podman[40317]: 2024-09-10 18:49:45.685022067 +0000 UTC m=+0.314899157 container init 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 podman[40317]: 2024-09-10 18:49:45.689608421 +0000 UTC m=+0.319485512 container start 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0) 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: set uid:gid to 167:167 (ceph:ceph) 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: ceph version 17.2.7-1606-g5be57395 (5be57395ea235535910512ae73ba8e7ff25fe588) quincy (stable), process ceph-mon, pid 7 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pidfile_write: ignore empty --pid-file 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: load: jerasure load: lrc 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: RocksDB version: 6.15.5 2024-09-10T18:49:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Git sha rocksdb_build_git_sha:@0@ 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Compile date Sep 9 2024 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: DB SUMMARY 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: DB Session ID: G4R7JLXVYHXZJRJHC5N7 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: CURRENT file: CURRENT 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: IDENTITY file: IDENTITY 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: MANIFEST file: MANIFEST-000003 size: 57 Bytes 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 636 ; 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.error_if_exists: 0 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.create_if_missing: 0 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.paranoid_checks: 1 2024-09-10T18:49:45.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.env: 0x5569f2369440 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.fs: Posix File System 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.info_log: 0x5569f491cec0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_file_opening_threads: 16 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.statistics: (nil) 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.use_fsync: 0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_log_file_size: 0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.keep_log_file_num: 1000 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.recycle_log_file_num: 0 2024-09-10T18:49:45.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_fallocate: 1 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_mmap_reads: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_mmap_writes: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.use_direct_reads: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.create_missing_column_families: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.db_log_dir: 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.wal_dir: /var/lib/ceph/mon/ceph-c/store.db 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-10T18:49:45.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.advise_random_on_open: 1 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.db_write_buffer_size: 0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.write_buffer_manager: 0x5569f49da7b0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.new_table_reader_for_compaction_inputs: 0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.rate_limiter: (nil) 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.wal_recovery_mode: 2 2024-09-10T18:49:45.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.enable_thread_tracking: 0 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.enable_pipelined_write: 0 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.unordered_write: 0 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.row_cache: None 2024-09-10T18:49:45.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.wal_filter: None 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_ingest_behind: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.preserve_deletes: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.two_write_queues: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.manual_wal_flush: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.atomic_flush: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-10T18:49:45.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.log_readahead_size: 0 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.best_efforts_recovery: 0 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.allow_data_in_errors: 0 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.db_host_id: __hostname__ 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_background_jobs: 2 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_background_compactions: -1 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_subcompactions: 1 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-10T18:49:45.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_total_wal_size: 0 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_open_files: -1 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bytes_per_sync: 0 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_readahead_size: 0 2024-09-10T18:49:45.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_background_flushes: -1 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Compression algorithms supported: 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kZSTD supported: 0 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kXpressCompression supported: 0 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kBZip2Compression supported: 0 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kLZ4Compression supported: 1 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kZlibCompression supported: 1 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kLZ4HCCompression supported: 1 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: kSnappyCompression supported: 1 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-10T18:49:45.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/version_set.cc:4724] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/column_family.cc:595] --------------- Options for column family [default]: 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.merge_operator: 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_filter: None 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_filter_factory: None 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.sst_partitioner_factory: None 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5569f48c3a38) 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks: 1 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-10T18:49:45.977 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: pin_top_level_index_and_filter: 1 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_type: 0 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_index_type: 0 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_shortening: 1 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: hash_index_allow_collision: 1 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: checksum: 1 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: no_block_cache: 0 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache: 0x5569f48f91f0 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_name: BinnedLRUCache 2024-09-10T18:49:45.978 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_options: 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: capacity : 536870912 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: num_shard_bits : 4 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: strict_capacity_limit : 0 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: high_pri_pool_ratio: 0.000 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_cache_compressed: (nil) 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: persistent_cache: (nil) 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size: 4096 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_size_deviation: 10 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_restart_interval: 16 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: index_block_restart_interval: 1 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: metadata_block_size: 4096 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: partition_filters: 0 2024-09-10T18:49:45.979 INFO:journalctl@ceph.mon.c.smithi107.stdout: use_delta_encoding: 1 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: filter_policy: rocksdb.BuiltinBloomFilter 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: whole_key_filtering: 1 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: verify_compression: 0 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: read_amp_bytes_per_bit: 0 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: format_version: 4 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_index_compression: 1 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout: block_align: 0 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.write_buffer_size: 33554432 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_write_buffer_number: 2 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression: NoCompression 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression: Disabled 2024-09-10T18:49:45.980 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.prefix_extractor: nullptr 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.num_levels: 7 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-10T18:49:45.981 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.level: 32767 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.strategy: 0 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compression_opts.enabled: false 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.target_file_size_base: 67108864 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-10T18:49:45.982 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.arena_block_size: 4194304 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.rate_limit_delay_max_milliseconds: 100 2024-09-10T18:49:45.983 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.disable_auto_compactions: 0 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.inplace_update_support: 0 2024-09-10T18:49:45.984 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.bloom_locality: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.max_successive_merges: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.paranoid_file_checks: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.force_consistency_checks: 1 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.report_bg_io_stats: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.ttl: 2592000 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.enable_blob_files: false 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.min_blob_size: 0 2024-09-10T18:49:45.985 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.blob_file_size: 268435456 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/version_set.cc:4764] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000003 succeeded,manifest_file_number is 3, next_file_number is 5, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/version_set.cc:4779] Column family [default] (ID 0), log number is 0 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/version_set.cc:4082] Creating manifest 7 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994185713544, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/db_impl/db_impl_open.cc:845] Recovering log #4 mode 2 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [table/block_based/filter_policy.cc:991] Using legacy Bloom filter with high (20) bits/key. Dramatic filter space and/or accuracy improvement is available with format_version>=5. 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994185713839, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1696, "file_checksum": "", "file_checksum_func_name": "Unknown", "table_properties": {"data_size": 648, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 526, "raw_average_value_size": 105, "num_data_blocks": 1, "num_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "rocksdb.BuiltinBloomFilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; ", "creation_time": 1725994185, "oldest_key_time": 0, "file_creation_time": 0, "db_id": "de21ccbf-26bb-4d95-909d-f555b9617479", "db_session_id": "G4R7JLXVYHXZJRJHC5N7"}} 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/version_set.cc:4082] Creating manifest 9 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: EVENT_LOG_v1 {"time_micros": 1725994185714168, "job": 1, "event": "recovery_finished"} 2024-09-10T18:49:45.986 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [file/delete_scheduler.cc:69] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/db_impl/db_impl_open.cc:1700] SstFileManager instance 0x5569f492c700 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: DB pointer 0x5569f48fc000 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/db_impl/db_impl.cc:901] ------- DUMPING STATS ------- 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: rocksdb: [db/db_impl/db_impl.cc:903] 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** DB Stats ** 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 MB, 0.00 MB/s 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-10T18:49:45.987 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.988 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval compaction: 0.00 GB write, 0.63 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:45.989 INFO:journalctl@ceph.mon.c.smithi107.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: L0 1/0 1.66 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: Sum 1/0 1.66 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.00 0.00 0 0.000 0 0 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** Compaction Stats [default] ** 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 5.7 0.00 0.00 1 0.000 0 0 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:45.990 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: Cumulative compaction: 0.00 GB write, 0.62 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: Interval compaction: 0.00 GB write, 0.00 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: using public_addrv [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: starting mon.c rank -1 at public addrs [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] at bind addrs [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] mon_data /var/lib/ceph/mon/ceph-c fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:45.991 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(???) e0 preinit fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).mds e1 new map 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: e1 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2} 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: legacy client fscid: -1 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout: No filesystems configured 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-10T18:49:45.992 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.a calling monitor election 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.993 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.b calling monitor election 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: monmap e2: 2 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: fsmap 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mgrmap e13: y(active, since 39s) 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: overall HEALTH_OK 2024-09-10T18:49:45.994 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: Deploying daemon mon.c on smithi107 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/3146971259' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 ceph-mon[40355]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 bash[40317]: 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 2024-09-10T18:49:45.995 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:45 smithi107 systemd[1]: Started Ceph mon.c for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:49:46.002 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.084+0000 7fcd76f3b640 1 -- 172.21.15.162:0/4054926110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd600034f0 msgr2=0x7fcd60005970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.084+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/4054926110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd600034f0 0x7fcd60005970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fcd700fb160 tx=0x7fcd6400b040 comp rx=0 tx=0).stop 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.084+0000 7fcd76f3b640 1 -- 172.21.15.162:0/4054926110 shutdown_connections 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.084+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/4054926110 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd600034f0 0x7fcd60005970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.084+0000 7fcd76f3b640 1 -- 172.21.15.162:0/4054926110 >> 172.21.15.162:0/4054926110 conn(0x7fcd700f6dd0 msgr2=0x7fcd700f91e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.085+0000 7fcd76f3b640 1 -- 172.21.15.162:0/4054926110 shutdown_connections 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.085+0000 7fcd76f3b640 1 -- 172.21.15.162:0/4054926110 wait complete. 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.085+0000 7fcd76f3b640 1 Processor -- start 2024-09-10T18:49:51.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.085+0000 7fcd76f3b640 1 -- start start 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: mon.a calling monitor election 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: mon.b calling monitor election 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.088 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: fsmap 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: mgrmap e13: y(active, since 47s) 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: mon.c (rank 2) addr [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] is down (out of quorum) 2024-09-10T18:49:51.089 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.090 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.090 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.090 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70112870 con 0x7fcd70112210 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70118b20 con 0x7fcd600034f0 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd76f3b640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd70118c60 con 0x7fcd7010c190 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:51.092 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:56436/0 (socket says 172.21.15.162:56436) 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.162:42748/0 (socket says 172.21.15.162:42748) 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 -- 172.21.15.162:0/1576076033 learned_addr learned my addr 172.21.15.162:0/1576076033 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd754d8640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 msgr2=0x7fcd70111cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 msgr2=0x7fcd70111cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd754d8640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 msgr2=0x7fcd7010c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd754d8640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 msgr2=0x7fcd7010c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd754d8640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd74cd7640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd754d8640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:49:51.093 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 msgr2=0x7fcd7010c600 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 msgr2=0x7fcd70111cd0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.086+0000 7fcd6ffff640 1 -- 172.21.15.162:0/1576076033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd64005040 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd6ffff640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00b600 tx=0x7fcd5c0090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c010670 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fcd5c010c10 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd70118ef0 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd5c00fad0 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.087+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd701193b0 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.088+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fcd5c0213a0 con 0x7fcd70112210 2024-09-10T18:49:51.094 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.088+0000 7fcd6d7fa640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 0x7fcd5403bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:51.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.088+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcd5c04cdf0 con 0x7fcd70112210 2024-09-10T18:49:51.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.088+0000 7fcd74cd7640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 0x7fcd5403bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:51.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.088+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd700fc970 con 0x7fcd70112210 2024-09-10T18:49:51.095 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.089+0000 7fcd74cd7640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 0x7fcd5403bee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd64007b60 tx=0x7fcd6402d040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:51.096 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.095+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcd5c017b30 con 0x7fcd70112210 2024-09-10T18:49:51.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: mon.a calling monitor election 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: mon.b calling monitor election 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:51.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: fsmap 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: mgrmap e13: y(active, since 47s) 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: mon.c (rank 2) addr [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] is down (out of quorum) 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:51.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.360+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) v1 -- 0x7fcd70102230 con 0x7fcd70112210 2024-09-10T18:49:51.361 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.360+0000 7fcd6d7fa640 1 -- 172.21.15.162:0/1576076033 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) v1 ==== 95+0+1273 (secure 0 0 0) 0x7fcd5c00ded0 con 0x7fcd70112210 2024-09-10T18:49:51.362 INFO:teuthology.orchestra.run.smithi162.stdout: 2024-09-10T18:49:51.362 INFO:teuthology.orchestra.run.smithi162.stdout:{"epoch":3,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","modified":"2024-09-10T18:49:45.734739Z","created":"2024-09-10T18:48:14.624206Z","min_mon_release":17,"min_mon_release_name":"quincy","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3300","nonce":0},{"type":"v1","addr":"172.21.15.107:6789","nonce":0}]},"addr":"172.21.15.107:6789/0","public_addr":"172.21.15.107:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:3300","nonce":0},{"type":"v1","addr":"172.21.15.162:6789","nonce":0}]},"addr":"172.21.15.162:6789/0","public_addr":"172.21.15.162:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:3301","nonce":0},{"type":"v1","addr":"172.21.15.107:6790","nonce":0}]},"addr":"172.21.15.107:6790/0","public_addr":"172.21.15.107:6790/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-10T18:49:51.362 INFO:teuthology.orchestra.run.smithi162.stderr:dumped monmap epoch 3 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.362+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 msgr2=0x7fcd5403bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.362+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 0x7fcd5403bee0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fcd64007b60 tx=0x7fcd6402d040 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.362+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 msgr2=0x7fcd7010bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.362+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fcd5c00b600 tx=0x7fcd5c0090f0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 shutdown_connections 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd54039a30 0x7fcd5403bee0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd7010c190 0x7fcd7010c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd70112210 0x7fcd7010bc50 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 --2- 172.21.15.162:0/1576076033 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd600034f0 0x7fcd70111cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 >> 172.21.15.162:0/1576076033 conn(0x7fcd700f6dd0 msgr2=0x7fcd70065770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:51.364 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 shutdown_connections 2024-09-10T18:49:51.365 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:51.363+0000 7fcd76f3b640 1 -- 172.21.15.162:0/1576076033 wait complete. 2024-09-10T18:49:51.957 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:51.957 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:51 smithi162 ceph-mon[33228]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:51.957 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/1576076033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:51.957 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:51.982 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-10T18:49:51.982 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph config generate-minimal-conf 2024-09-10T18:49:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:51 smithi107 ceph-mon[29404]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/1576076033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.216 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: mon.a calling monitor election 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: mon.b calling monitor election 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: mon.a is new leader, mons a,b in quorum (ranks 0,1) 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: fsmap 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: mgrmap e13: y(active, since 47s) 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: Health check failed: 1/3 mons down, quorum a,b (MON_DOWN) 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: Health detail: HEALTH_WARN 1/3 mons down, quorum a,b 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: [WRN] MON_DOWN: 1/3 mons down, quorum a,b 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: mon.c (rank 2) addr [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] is down (out of quorum) 2024-09-10T18:49:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/1576076033' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-09-10T18:49:52.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mon.c calling monitor election 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mon.c calling monitor election 2024-09-10T18:49:53.614 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mon.b calling monitor election 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mon.a calling monitor election 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: fsmap 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: mgrmap e13: y(active, since 49s) 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: Cluster is now healthy 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: overall HEALTH_OK 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.615 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mon.c calling monitor election 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mon.c calling monitor election 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mon.b calling monitor election 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mon.a calling monitor election 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: fsmap 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: mgrmap e13: y(active, since 49s) 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: Cluster is now healthy 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: overall HEALTH_OK 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:53.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:53.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:53.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:53 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mon.c calling monitor election 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mon.c calling monitor election 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mon.b calling monitor election 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mon.a calling monitor election 2024-09-10T18:49:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mon.a is new leader, mons a,b,c in quorum (ranks 0,1,2) 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: monmap e3: 3 mons at {a=[v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0],b=[v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0],c=[v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0]} removed_ranks: {} disallowed_leaders: {} 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: fsmap 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: osdmap e4: 0 total, 0 up, 0 in 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: mgrmap e13: y(active, since 49s) 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Health check cleared: MON_DOWN (was: 1/3 mons down, quorum a,b) 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: Cluster is now healthy 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: overall HEALTH_OK 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:53 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- 172.21.15.107:0/1234764508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 msgr2=0x7fac34102620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 --2- 172.21.15.107:0/1234764508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34102620 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fac24009bd0 tx=0x7fac24004d10 comp rx=0 tx=0).stop 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- 172.21.15.107:0/1234764508 shutdown_connections 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 --2- 172.21.15.107:0/1234764508 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34102620 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- 172.21.15.107:0/1234764508 >> 172.21.15.107:0/1234764508 conn(0x7fac34071790 msgr2=0x7fac34071b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- 172.21.15.107:0/1234764508 shutdown_connections 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- 172.21.15.107:0/1234764508 wait complete. 2024-09-10T18:49:54.397 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 Processor -- start 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.389+0000 7fac39682640 1 -- start start 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac341123c0 0x7fac3410be00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 0x7fac3410c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34112a20 con 0x7fac34102250 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac34118cd0 con 0x7fac341123c0 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac34118e10 con 0x7fac3410c340 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52574/0 (socket says 172.21.15.107:52574) 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 -- 172.21.15.107:0/2939260336 learned_addr learned my addr 172.21.15.107:0/2939260336 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac337fe640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 0x7fac3410c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 -- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 msgr2=0x7fac3410c7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:54.398 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 0x7fac3410c7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 -- 172.21.15.107:0/2939260336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac341123c0 msgr2=0x7fac3410be00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac341123c0 0x7fac3410be00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 -- 172.21.15.107:0/2939260336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240098b0 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac337fe640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 0x7fac3410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac32ffd640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ba0 tx=0x7fac24004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac24032430 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fac24033d20 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac34119040 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.390+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac2403d470 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.391+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac341195e0 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.391+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7fac2402f030 con 0x7fac34102250 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.391+0000 7fac2b7fe640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 0x7fac2003bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:54.399 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.391+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac34064750 con 0x7fac34102250 2024-09-10T18:49:54.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.392+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fac2406cf30 con 0x7fac34102250 2024-09-10T18:49:54.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.392+0000 7fac327fc640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 0x7fac2003bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:54.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.392+0000 7fac327fc640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 0x7fac2003bee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006fd0 tx=0x7fac1c006d50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:54.400 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.396+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fac24045b90 con 0x7fac34102250 2024-09-10T18:49:54.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.554+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) v1 -- 0x7fac3410d520 con 0x7fac34102250 2024-09-10T18:49:54.556 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.555+0000 7fac2b7fe640 1 -- 172.21.15.107:0/2939260336 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) v1 ==== 76+0+277 (secure 0 0 0) 0x7fac2405f9c0 con 0x7fac34102250 2024-09-10T18:49:54.556 INFO:teuthology.orchestra.run.smithi107.stdout:# minimal ceph.conf for d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:54.557 INFO:teuthology.orchestra.run.smithi107.stdout:[global] 2024-09-10T18:49:54.557 INFO:teuthology.orchestra.run.smithi107.stdout: fsid = d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:49:54.557 INFO:teuthology.orchestra.run.smithi107.stdout: mon_host = [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] 2024-09-10T18:49:54.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 msgr2=0x7fac2003bee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:54.558 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 0x7fac2003bee0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fac1c006fd0 tx=0x7fac1c006d50 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 msgr2=0x7fac34111e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fac24009ba0 tx=0x7fac24004d10 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 shutdown_connections 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fac20039a30 0x7fac2003bee0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fac3410c340 0x7fac3410c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fac341123c0 0x7fac3410be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 --2- 172.21.15.107:0/2939260336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fac34102250 0x7fac34111e80 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 >> 172.21.15.107:0/2939260336 conn(0x7fac34071790 msgr2=0x7fac34102f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 shutdown_connections 2024-09-10T18:49:54.559 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:49:54.557+0000 7fac39682640 1 -- 172.21.15.107:0/2939260336 wait complete. 2024-09-10T18:49:54.567 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[40355]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:49:54.567 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[40355]: Reconfiguring daemon mon.a on smithi107 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[40355]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[29404]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[29404]: Reconfiguring daemon mon.a on smithi107 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[29404]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:54.568 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:54 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:54 smithi162 ceph-mon[33228]: Reconfiguring mon.a (unknown last config time)... 2024-09-10T18:49:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:54 smithi162 ceph-mon[33228]: Reconfiguring daemon mon.a on smithi107 2024-09-10T18:49:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:54 smithi162 ceph-mon[33228]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:54 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:49:55.552 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-10T18:49:55.552 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:55.552 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2939260336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:55.575 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2939260336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:55.576 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:55 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:55.580 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:55.580 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:55.647 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:49:55.647 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.conf 2024-09-10T18:49:55.680 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:49:55.680 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:49:55.749 INFO:tasks.cephadm:Adding mgr.y on smithi107 2024-09-10T18:49:55.749 INFO:tasks.cephadm:Adding mgr.x on smithi162 2024-09-10T18:49:55.749 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply mgr '2;smithi107=y;smithi162=x' 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2939260336' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:55.813 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:55 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:56.027 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:49:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[40355]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:49:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[40355]: Reconfiguring daemon mon.c on smithi107 2024-09-10T18:49:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[40355]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[29404]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:49:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[29404]: Reconfiguring daemon mon.c on smithi107 2024-09-10T18:49:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:56 smithi107 ceph-mon[29404]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:56.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:56 smithi162 ceph-mon[33228]: Reconfiguring mon.c (monmap changed)... 2024-09-10T18:49:56.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:56 smithi162 ceph-mon[33228]: Reconfiguring daemon mon.c on smithi107 2024-09-10T18:49:56.831 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:56 smithi162 ceph-mon[33228]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:58.072 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: Reconfiguring daemon mon.b on smithi162 2024-09-10T18:49:58.073 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:57 smithi162 ceph-mon[33228]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: Reconfiguring daemon mon.b on smithi162 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[29404]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: Reconfiguring mon.b (monmap changed)... 2024-09-10T18:49:58.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-09-10T18:49:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-09-10T18:49:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: Reconfiguring daemon mon.b on smithi162 2024-09-10T18:49:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:57 smithi107 ceph-mon[40355]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.896+0000 7f26d5a12640 1 -- 172.21.15.162:0/2611134384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 msgr2=0x7f26d00fef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.896+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2611134384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d00fef60 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f26c0004850 tx=0x7f26c0009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 -- 172.21.15.162:0/2611134384 shutdown_connections 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2611134384 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d00fef60 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 -- 172.21.15.162:0/2611134384 >> 172.21.15.162:0/2611134384 conn(0x7f26d00f69b0 msgr2=0x7f26d00f8de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 -- 172.21.15.162:0/2611134384 shutdown_connections 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 -- 172.21.15.162:0/2611134384 wait complete. 2024-09-10T18:49:58.549 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 Processor -- start 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled mgr update... 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 -- start start 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d010bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.897+0000 7f26d5a12640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d010c440 0x7f26d010d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d010ca60 con 0x7f26d00663d0 2024-09-10T18:49:58.550 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01143f0 con 0x7f26d010d6b0 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26d01145b0 con 0x7f26d010c440 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:32972/0 (socket says 172.21.15.162:32972) 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 -- 172.21.15.162:0/2111443901 learned_addr learned my addr 172.21.15.162:0/2111443901 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26ceffd640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d010bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26ce7fc640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d010c440 0x7f26d010d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 -- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d010c440 msgr2=0x7f26d010d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d010c440 0x7f26d010d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 -- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 msgr2=0x7f26d010bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d010bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 -- 172.21.15.162:0/2111443901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26c0005040 con 0x7f26d010d6b0 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26ceffd640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d010bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:49:58.551 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26cf7fe640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26c800bc30 tx=0x7f26c8009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c800ca70 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f26c800cbd0 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26d0114890 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26c80073e0 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.898+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26d0114de0 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.899+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 13) v1 ==== 45549+0+0 (secure 0 0 0) 0x7f26c8007810 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.900+0000 7f26c77fe640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 0x7f26bc03be90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.900+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7f26c804c1d0 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.900+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26d005ed70 con 0x7f26d010d6b0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.900+0000 7f26ceffd640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 0x7f26bc03be90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.900+0000 7f26ceffd640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 0x7f26bc03be90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007fd0 tx=0x7f26c0009da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:49:58.552 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:57.905+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f26c80112c0 con 0x7f26d010d6b0 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.070+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi162=x", "target": ["mon-mgr", ""]}) v1 -- 0x7f26d0101280 con 0x7f26bc0399e0 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.077+0000 7f26c77fe640 1 -- 172.21.15.162:0/2111443901 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+24 (secure 0 0 0) 0x7f26d0101280 con 0x7f26bc0399e0 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 msgr2=0x7f26bc03be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 0x7f26bc03be90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f26c0007fd0 tx=0x7f26c0009da0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 msgr2=0x7f26d010db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f26c800bc30 tx=0x7f26c8009820 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 shutdown_connections 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f26bc0399e0 0x7f26bc03be90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f26d010d6b0 0x7f26d010db20 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f26d010c440 0x7f26d010d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 --2- 172.21.15.162:0/2111443901 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f26d00663d0 0x7f26d010bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:49:58.553 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 >> 172.21.15.162:0/2111443901 conn(0x7f26d00f69b0 msgr2=0x7f26d00f7590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:49:58.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 shutdown_connections 2024-09-10T18:49:58.554 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:49:58.079+0000 7f26d5a12640 1 -- 172.21.15.162:0/2111443901 wait complete. 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi162=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: Saving service mgr spec with placement smithi107=y;smithi162=x;count:2 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.099 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.100 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:49:59 smithi162 ceph-mon[33228]: Deploying daemon mgr.x on smithi162 2024-09-10T18:49:59.101 DEBUG:teuthology.orchestra.run.smithi162:mgr.x> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.x.service 2024-09-10T18:49:59.143 INFO:tasks.cephadm:Deploying OSDs... 2024-09-10T18:49:59.143 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:49:59.143 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:49:59.163 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:49:59.163 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_1 2024-09-10T18:49:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:49:59.221 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:49:59.221 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 982 Links: 1 2024-09-10T18:49:59.221 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:49:59.222 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:49:59.222 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 18:49:45.212461534 +0000 2024-09-10T18:49:59.222 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 18:43:18.459791464 +0000 2024-09-10T18:49:59.222 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 18:43:18.459791464 +0000 2024-09-10T18:49:59.222 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 18:43:18.459791464 +0000 2024-09-10T18:49:59.222 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:49:59.290 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T18:49:59.290 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T18:49:59.290 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000229905 s, 2.2 MB/s 2024-09-10T18:49:59.291 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:49:59.350 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_2 2024-09-10T18:49:59.407 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:49:59.407 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:49:59.407 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 973 Links: 1 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 18:49:45.213461518 +0000 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 18:43:18.457790977 +0000 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 18:43:18.457790977 +0000 2024-09-10T18:49:59.408 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 18:43:18.457790977 +0000 2024-09-10T18:49:59.408 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:49:59.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi162=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:59.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: Saving service mgr spec with placement smithi107=y;smithi162=x;count:2 2024-09-10T18:49:59.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[40355]: Deploying daemon mgr.x on smithi162 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "2;smithi107=y;smithi162=x", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:49:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: Saving service mgr spec with placement smithi107=y;smithi162=x;count:2 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:49:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:49:59 smithi107 ceph-mon[29404]: Deploying daemon mgr.x on smithi162 2024-09-10T18:49:59.473 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T18:49:59.473 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T18:49:59.473 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000202351 s, 2.5 MB/s 2024-09-10T18:49:59.475 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:49:59.533 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_3 2024-09-10T18:49:59.590 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:49:59.590 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:49:59.590 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 966 Links: 1 2024-09-10T18:49:59.590 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:49:59.591 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:49:59.591 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 18:49:45.213461518 +0000 2024-09-10T18:49:59.591 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 18:43:18.456790733 +0000 2024-09-10T18:49:59.591 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 18:43:18.456790733 +0000 2024-09-10T18:49:59.591 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 18:43:18.456790733 +0000 2024-09-10T18:49:59.591 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:49:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T18:49:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T18:49:59.655 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.00023652 s, 2.2 MB/s 2024-09-10T18:49:59.656 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:49:59.713 DEBUG:teuthology.orchestra.run.smithi107:> stat /dev/vg_nvme/lv_4 2024-09-10T18:49:59.769 INFO:teuthology.orchestra.run.smithi107.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:49:59.769 INFO:teuthology.orchestra.run.smithi107.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:49:59.769 INFO:teuthology.orchestra.run.smithi107.stdout:Device: 5h/5d Inode: 977 Links: 1 2024-09-10T18:49:59.769 INFO:teuthology.orchestra.run.smithi107.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:49:59.770 INFO:teuthology.orchestra.run.smithi107.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:49:59.770 INFO:teuthology.orchestra.run.smithi107.stdout:Access: 2024-09-10 18:49:45.213461518 +0000 2024-09-10T18:49:59.770 INFO:teuthology.orchestra.run.smithi107.stdout:Modify: 2024-09-10 18:43:18.458791221 +0000 2024-09-10T18:49:59.770 INFO:teuthology.orchestra.run.smithi107.stdout:Change: 2024-09-10 18:43:18.458791221 +0000 2024-09-10T18:49:59.770 INFO:teuthology.orchestra.run.smithi107.stdout: Birth: 2024-09-10 18:43:18.458791221 +0000 2024-09-10T18:49:59.770 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:49:59.833 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records in 2024-09-10T18:49:59.833 INFO:teuthology.orchestra.run.smithi107.stderr:1+0 records out 2024-09-10T18:49:59.833 INFO:teuthology.orchestra.run.smithi107.stderr:512 bytes copied, 0.000188368 s, 2.7 MB/s 2024-09-10T18:49:59.834 DEBUG:teuthology.orchestra.run.smithi107:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:49:59.891 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:49:59.891 DEBUG:teuthology.orchestra.run.smithi162:> dd if=/scratch_devs of=/dev/stdout 2024-09-10T18:49:59.908 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-10T18:49:59.909 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_1 2024-09-10T18:49:59.964 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-10T18:49:59.964 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:49:59.964 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 961 Links: 1 2024-09-10T18:49:59.964 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:49:59.965 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:49:59.965 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 18:49:36.338624479 +0000 2024-09-10T18:49:59.965 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 18:43:18.341141063 +0000 2024-09-10T18:49:59.965 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 18:43:18.341141063 +0000 2024-09-10T18:49:59.965 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 18:43:18.341141063 +0000 2024-09-10T18:49:59.965 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-10T18:50:00.029 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T18:50:00.029 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T18:50:00.030 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000216341 s, 2.4 MB/s 2024-09-10T18:50:00.031 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-10T18:50:00.087 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_2 2024-09-10T18:50:00.143 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-10T18:50:00.143 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:50:00.143 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 983 Links: 1 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 18:49:36.339624459 +0000 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.144 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.144 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-10T18:50:00.205 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:00 smithi162 ceph-mon[33228]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:00.205 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:00 smithi162 ceph-mon[33228]: overall HEALTH_OK 2024-09-10T18:50:00.207 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T18:50:00.208 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T18:50:00.208 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000191178 s, 2.7 MB/s 2024-09-10T18:50:00.209 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-10T18:50:00.265 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_3 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 980 Links: 1 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 18:49:36.339624459 +0000 2024-09-10T18:50:00.322 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.323 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.323 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 18:43:18.343141129 +0000 2024-09-10T18:50:00.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-10T18:50:00.420 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T18:50:00.420 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T18:50:00.420 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000199508 s, 2.6 MB/s 2024-09-10T18:50:00.421 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-10T18:50:00.437 DEBUG:teuthology.orchestra.run.smithi162:> stat /dev/vg_nvme/lv_4 2024-09-10T18:50:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:00 smithi107 ceph-mon[29404]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:00 smithi107 ceph-mon[29404]: overall HEALTH_OK 2024-09-10T18:50:00.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:00 smithi107 ceph-mon[40355]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:00.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:00 smithi107 ceph-mon[40355]: overall HEALTH_OK 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Device: 5h/5d Inode: 970 Links: 1 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Context: system_u:object_r:device_t:s0 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Access: 2024-09-10 18:50:00.370146014 +0000 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Modify: 2024-09-10 18:43:18.342141096 +0000 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout:Change: 2024-09-10 18:43:18.342141096 +0000 2024-09-10T18:50:00.499 INFO:teuthology.orchestra.run.smithi162.stdout: Birth: 2024-09-10 18:43:18.342141096 +0000 2024-09-10T18:50:00.500 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-10T18:50:00.614 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records in 2024-09-10T18:50:00.614 INFO:teuthology.orchestra.run.smithi162.stderr:1+0 records out 2024-09-10T18:50:00.614 INFO:teuthology.orchestra.run.smithi162.stderr:512 bytes copied, 0.000182568 s, 2.8 MB/s 2024-09-10T18:50:00.616 DEBUG:teuthology.orchestra.run.smithi162:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-10T18:50:00.631 INFO:tasks.cephadm:Deploying osd.0 on smithi107 with /dev/vg_nvme/lv_4... 2024-09-10T18:50:00.632 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:50:00.826 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:00 smithi162 systemd[1]: Starting Ceph mgr.x for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:50:00.871 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:01.153 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:00 smithi162 podman[35274]: 2024-09-10 18:50:00.825445186 +0000 UTC m=+0.099679509 container create 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True) 2024-09-10T18:50:01.153 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:00 smithi162 podman[35274]: 2024-09-10 18:50:00.741695892 +0000 UTC m=+0.015930224 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:50:01.153 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 podman[35274]: 2024-09-10 18:50:01.044285813 +0000 UTC m=+0.318520136 container init 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-09-10T18:50:01.153 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 podman[35274]: 2024-09-10 18:50:01.04937332 +0000 UTC m=+0.323607644 container start 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-10T18:50:01.154 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-mgr[35312]: -- 172.21.15.162:0/1985300322 <== mon.1 v2:172.21.15.162:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x558b3c614340 con 0x558b3c620c00 2024-09-10T18:50:01.154 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 bash[35274]: 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f 2024-09-10T18:50:01.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 systemd[1]: Started Ceph mgr.x for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:50:01.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.165+0000 7ffbfa933ec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:50:01.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.224+0000 7ffbfa933ec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:50:01.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.292+0000 7ffbfa933ec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:50:01.848 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.424+0000 7ffbfa933ec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:50:01.848 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.571+0000 7ffbfa933ec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:50:01.848 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.847+0000 7ffbfa933ec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:50:02.131 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:01.910+0000 7ffbfa933ec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:50:02.132 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:02.080+0000 7ffbfa933ec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:02.565 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:50:02.665 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:50:02.681 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-10T18:50:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-10T18:50:02.681 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0135381 s, 775 MB/s 2024-09-10T18:50:02.681 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-10T18:50:03.337 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_4 2024-09-10T18:50:03.413 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:02.960+0000 7ffbfa933ec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:50:03.414 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:03.023+0000 7ffbfa933ec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:50:03.634 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: Reconfiguring daemon mgr.y on smithi107 2024-09-10T18:50:03.635 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: Reconfiguring daemon mgr.y on smithi107 2024-09-10T18:50:03.636 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: Reconfiguring mgr.y (unknown last config time)... 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr services"}]: dispatch 2024-09-10T18:50:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:03.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: Reconfiguring daemon mgr.y on smithi107 2024-09-10T18:50:03.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:04.024 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:03.735+0000 7ffbfa933ec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:50:04.024 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:03.845+0000 7ffbfa933ec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:50:04.024 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:03.964+0000 7ffbfa933ec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:50:04.025 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.023+0000 7ffbfa933ec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:50:04.089 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:04.401 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.147+0000 7ffbfa933ec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:50:04.402 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.212+0000 7ffbfa933ec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:50:04.402 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.268+0000 7ffbfa933ec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:50:04.402 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.400+0000 7ffbfa933ec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:50:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:04 smithi107 ceph-mon[29404]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:04 smithi107 ceph-mon[40355]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:04.666 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-mon[33228]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:04.666 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.465+0000 7ffbfa933ec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:50:04.667 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.528+0000 7ffbfa933ec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:50:04.667 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.608+0000 7ffbfa933ec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:50:04.667 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.665+0000 7ffbfa933ec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:50:04.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:04 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:04.722+0000 7ffbfa933ec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:50:05.762 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:50:05.469+0000 7ffbfa933ec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: Standby manager daemon x started 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:50:05.887 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:05.888 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: Standby manager daemon x started 2024-09-10T18:50:05.889 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:50:05.889 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:50:05.889 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:50:05.889 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:05 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- 172.21.15.107:0/881918076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fe7f0 msgr2=0x7fcffc0fa950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 --2- 172.21.15.107:0/881918076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fa950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fcff00048c0 tx=0x7fcff0009da0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- 172.21.15.107:0/881918076 shutdown_connections 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 --2- 172.21.15.107:0/881918076 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fb080 0x7fcffc104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 --2- 172.21.15.107:0/881918076 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fa950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 --2- 172.21.15.107:0/881918076 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fdea0 0x7fcffc0fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- 172.21.15.107:0/881918076 >> 172.21.15.107:0/881918076 conn(0x7fcffc0f66a0 msgr2=0x7fcffc0f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- 172.21.15.107:0/881918076 shutdown_connections 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- 172.21.15.107:0/881918076 wait complete. 2024-09-10T18:50:06.095 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 Processor -- start 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.082+0000 7fd0012c2640 1 -- start start 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fb080 0x7fcffc0fd4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fc0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0fc800 con 0x7fcffc0fdea0 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0fc970 con 0x7fcffc0fe7f0 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc116ce0 con 0x7fcffc0fb080 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:46020/0 (socket says 172.21.15.107:46020) 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 -- 172.21.15.107:0/2454003894 learned_addr learned my addr 172.21.15.107:0/2454003894 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffaffd640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fb080 0x7fcffc0fd4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:06.096 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffb7fe640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fc0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 -- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fb080 msgr2=0x7fcffc0fd4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fb080 0x7fcffc0fd4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 -- 172.21.15.107:0/2454003894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fe7f0 msgr2=0x7fcffc0fc0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fc0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 -- 172.21.15.107:0/2454003894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfec0098b0 con 0x7fcffc0fdea0 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcffa7fc640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcff002d040 tx=0x7fcff0035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0009320 con 0x7fcffc0fdea0 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fcff0009480 con 0x7fcffc0fdea0 2024-09-10T18:50:06.097 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff0005040 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff003c680 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.083+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc117130 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.084+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc04bf60 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.085+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 90725+0+0 (secure 0 0 0) 0x7fcff0030020 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.085+0000 7fcfe37fe640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 0x7fcfe4071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.085+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 1151+0+0 (secure 0 0 0) 0x7fcff00ad270 con 0x7fcffc0fdea0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.085+0000 7fcffaffd640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 0x7fcfe4071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.086+0000 7fcffaffd640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 0x7fcfe4071240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfec004fd0 tx=0x7fcfec004d10 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:06.098 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.089+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcff0077810 con 0x7fcffc0fdea0 2024-09-10T18:50:06.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:06.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: Standby manager daemon x started 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:50:06.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:05 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/363946861' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:50:06.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:06.253+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fcffc107580 con 0x7fcfe406ed90 2024-09-10T18:50:06.785 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:06.786 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:06.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:06.787 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:06 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: mgrmap e14: y(active, since 62s), standbys: x 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: from='client.14208 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:06 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:07.890 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:07 smithi107 ceph-mon[29404]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:07.891 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:07 smithi107 ceph-mon[40355]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:07 smithi162 ceph-mon[33228]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:09.095 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/38696618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]': finished 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[29404]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/38696618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]': finished 2024-09-10T18:50:09.097 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[40355]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:50:09.097 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/38696618' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]: dispatch 2024-09-10T18:50:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1106f881-aeec-499d-9e94-6d0e2a28a6d1"}]': finished 2024-09-10T18:50:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:08 smithi162 ceph-mon[33228]: osdmap e5: 1 total, 0 up, 1 in 2024-09-10T18:50:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1197805947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:09 smithi162 ceph-mon[33228]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:10.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1197805947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:10.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:09 smithi107 ceph-mon[29404]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1197805947' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:09 smithi107 ceph-mon[40355]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:12.449 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:12 smithi107 ceph-mon[29404]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:12.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:12 smithi107 ceph-mon[40355]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:12 smithi162 ceph-mon[33228]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:14 smithi107 ceph-mon[29404]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:14 smithi107 ceph-mon[40355]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:14 smithi162 ceph-mon[33228]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:15.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:50:15.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:15.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[29404]: Deploying daemon osd.0 on smithi107 2024-09-10T18:50:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:50:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:15 smithi107 ceph-mon[40355]: Deploying daemon osd.0 on smithi107 2024-09-10T18:50:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:15 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-09-10T18:50:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:15 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:15 smithi162 ceph-mon[33228]: Deploying daemon osd.0 on smithi107 2024-09-10T18:50:16.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:16 smithi107 ceph-mon[29404]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:16.416 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:16 smithi107 ceph-mon[40355]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:16 smithi162 ceph-mon[33228]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:18.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:18 smithi107 ceph-mon[29404]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:18.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:18 smithi107 ceph-mon[40355]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:18 smithi162 ceph-mon[33228]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:20.458 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[29404]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[40355]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:20.459 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:20 smithi162 ceph-mon[33228]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:22.385 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:22 smithi107 ceph-mon[29404]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:22.386 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:22 smithi107 ceph-mon[40355]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:22.419 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.417+0000 7fcfe37fe640 1 -- 172.21.15.107:0/2454003894 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fcffc107580 con 0x7fcfe406ed90 2024-09-10T18:50:22.419 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 0 on host 'smithi107' 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 msgr2=0x7fcfe4071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 0x7fcfe4071240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcfec004fd0 tx=0x7fcfec004d10 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 msgr2=0x7fcffc0fbb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fcff002d040 tx=0x7fcff0035040 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 shutdown_connections 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcfe406ed90 0x7fcfe4071240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcffc0fe7f0 0x7fcffc0fc0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcffc0fdea0 0x7fcffc0fbb80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 --2- 172.21.15.107:0/2454003894 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcffc0fb080 0x7fcffc0fd4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 >> 172.21.15.107:0/2454003894 conn(0x7fcffc0f66a0 msgr2=0x7fcffc100ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:50:22.421 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 shutdown_connections 2024-09-10T18:50:22.422 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:22.419+0000 7fd0012c2640 1 -- 172.21.15.107:0/2454003894 wait complete. 2024-09-10T18:50:22.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:22 smithi162 ceph-mon[33228]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:23.015 DEBUG:teuthology.orchestra.run.smithi107:osd.0> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.0.service 2024-09-10T18:50:23.017 INFO:tasks.cephadm:Deploying osd.1 on smithi107 with /dev/vg_nvme/lv_3... 2024-09-10T18:50:23.017 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:50:23.313 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:23.666 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.667 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.668 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.668 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.668 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-09-10T18:50:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[29404]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:50:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:50:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-mon[40355]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:24.717 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T18:50:24.426+0000 7f7939966640 -1 osd.0 0 waiting for initial osdmap 2024-09-10T18:50:24.717 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:50:24 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T18:50:24.429+0000 7f793479d640 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: osdmap e6: 1 total, 0 up, 1 in 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: from='osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:24 smithi162 ceph-mon[33228]: from='osd.0 ' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:25.633 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[29404]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[29404]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[40355]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[40355]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:25.671 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:25 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:25.725 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:50:25.740 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-10T18:50:25.740 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-10T18:50:25.740 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132137 s, 794 MB/s 2024-09-10T18:50:25.741 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-10T18:50:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:25 smithi162 ceph-mon[33228]: from='osd.0 ' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:25 smithi162 ceph-mon[33228]: osdmap e7: 1 total, 0 up, 1 in 2024-09-10T18:50:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:25 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:25 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:50:26.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:50:26.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] boot 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:26.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.474 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] boot 2024-09-10T18:50:26.474 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:50:26.474 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:26 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: osd.0 [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] boot 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: osdmap e8: 1 total, 1 up, 1 in 2024-09-10T18:50:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:26 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:50:27.043 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_3 2024-09-10T18:50:27.672 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:27 smithi107 ceph-mon[40355]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:50:27.673 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:27 smithi107 ceph-mon[29404]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:50:27.797 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:27.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:27 smithi162 ceph-mon[33228]: osdmap e9: 1 total, 1 up, 1 in 2024-09-10T18:50:28.904 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:28.904 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi107 2024-09-10T18:50:28.904 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.904 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.904 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.905 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi107 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:28.906 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:28.907 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: pgmap v41: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi107 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 16063M 2024-09-10T18:50:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:29.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:29.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 -- 172.21.15.107:0/776589444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fb740 msgr2=0x7f8e300fbb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:29.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 --2- 172.21.15.107:0/776589444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fb740 0x7f8e300fbb50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c004950 tx=0x7f8e2c009da0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 -- 172.21.15.107:0/776589444 shutdown_connections 2024-09-10T18:50:29.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 --2- 172.21.15.107:0/776589444 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fd3c0 0x7f8e30103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.374 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 --2- 172.21.15.107:0/776589444 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 0x7f8e300fce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 --2- 172.21.15.107:0/776589444 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fb740 0x7f8e300fbb50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 -- 172.21.15.107:0/776589444 >> 172.21.15.107:0/776589444 conn(0x7f8e300f6f20 msgr2=0x7f8e300f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 -- 172.21.15.107:0/776589444 shutdown_connections 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.365+0000 7f8e38612640 1 -- 172.21.15.107:0/776589444 wait complete. 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 Processor -- start 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 -- start start 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 0x7f8e30111960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 0x7f8e30113ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e30114680 con 0x7f8e300fb740 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301147f0 con 0x7f8e300fca10 2024-09-10T18:50:29.375 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e38612640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e301149b0 con 0x7f8e300fd3c0 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e363ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e363ae640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51630/0 (socket says 172.21.15.107:51630) 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e363ae640 1 -- 172.21.15.107:0/763774293 learned_addr learned my addr 172.21.15.107:0/763774293 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e36baf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 0x7f8e30113ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.366+0000 7f8e35bad640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 0x7f8e30111960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 -- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 msgr2=0x7f8e30113ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 0x7f8e30113ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 -- 172.21.15.107:0/763774293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 msgr2=0x7f8e30111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 0x7f8e30111960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 -- 172.21.15.107:0/763774293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e280098b0 con 0x7f8e300fb740 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e36baf640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 0x7f8e30113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e363ae640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c02a570 tx=0x7f8e2c035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:29.376 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c033420 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8e2c0339c0 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e2c005040 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e2c03c660 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.367+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e30114fc0 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.368+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e3004bf60 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.369+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 90725+0+0 (secure 0 0 0) 0x7f8e2c030020 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.369+0000 7f8e26ffd640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 0x7f8e1c071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.369+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1763+0+0 (secure 0 0 0) 0x7f8e2c0acd50 con 0x7f8e300fb740 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.369+0000 7f8e35bad640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 0x7f8e1c071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:29.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.369+0000 7f8e35bad640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 0x7f8e1c071240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e300667b0 tx=0x7f8e20009040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:29.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.373+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8e2c0b0030 con 0x7f8e300fb740 2024-09-10T18:50:29.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:29.540+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e30106510 con 0x7f8e1c06ed90 2024-09-10T18:50:29.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:29.943 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:29.944 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:29.944 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:29.944 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:29.944 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:29 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:29 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:29 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:29 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:30.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:30 smithi107 ceph-mon[29404]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:30.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:30 smithi107 ceph-mon[29404]: from='client.14235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:30.930 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:30 smithi107 ceph-mon[40355]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:30.930 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:30 smithi107 ceph-mon[40355]: from='client.14235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:31.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:30 smithi162 ceph-mon[33228]: pgmap v42: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:31.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:30 smithi162 ceph-mon[33228]: from='client.14235 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]: dispatch 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]': finished 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3217403747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]: dispatch 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]': finished 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:32.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3217403747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: pgmap v43: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]: dispatch 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/937487320' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "7c834ad4-a6e2-408c-87fa-2953804ec22c"}]': finished 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: osdmap e10: 2 total, 1 up, 2 in 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:33.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3217403747' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:34.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:34 smithi107 ceph-mon[29404]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:34 smithi107 ceph-mon[40355]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:35.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:34 smithi162 ceph-mon[33228]: pgmap v45: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:36.930 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:36 smithi107 ceph-mon[29404]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:36.930 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:36 smithi107 ceph-mon[40355]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:37.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:36 smithi162 ceph-mon[33228]: pgmap v46: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:38.948 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[40355]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:38.948 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:50:38.948 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:38.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[29404]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:38.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:50:38.949 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:38 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:38 smithi162 ceph-mon[33228]: pgmap v47: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:38 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-09-10T18:50:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:38 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:39 smithi107 ceph-mon[29404]: Deploying daemon osd.1 on smithi107 2024-09-10T18:50:39.794 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:39 smithi107 ceph-mon[40355]: Deploying daemon osd.1 on smithi107 2024-09-10T18:50:40.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:39 smithi162 ceph-mon[33228]: Deploying daemon osd.1 on smithi107 2024-09-10T18:50:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:40 smithi107 ceph-mon[29404]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:40.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:40 smithi107 ceph-mon[40355]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:41.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:40 smithi162 ceph-mon[33228]: pgmap v48: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[29404]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[40355]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:42 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:43.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:42 smithi162 ceph-mon[33228]: pgmap v49: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:43.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:43.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:43.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:44 smithi107 ceph-mon[29404]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:44 smithi107 ceph-mon[40355]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:45.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:44 smithi162 ceph-mon[33228]: pgmap v50: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:45.980 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.978+0000 7f8e26ffd640 1 -- 172.21.15.107:0/763774293 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f8e30106510 con 0x7f8e1c06ed90 2024-09-10T18:50:45.980 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 1 on host 'smithi107' 2024-09-10T18:50:45.981 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 msgr2=0x7f8e1c071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 0x7f8e1c071240 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f8e300667b0 tx=0x7f8e20009040 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 msgr2=0x7f8e30066a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8e2c02a570 tx=0x7f8e2c035040 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 shutdown_connections 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8e1c06ed90 0x7f8e1c071240 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8e300fd3c0 0x7f8e30113ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8e300fca10 0x7f8e30111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 --2- 172.21.15.107:0/763774293 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8e300fb740 0x7f8e30066a00 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 >> 172.21.15.107:0/763774293 conn(0x7f8e300f6f20 msgr2=0x7f8e300fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 shutdown_connections 2024-09-10T18:50:45.982 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:45.980+0000 7f8e38612640 1 -- 172.21.15.107:0/763774293 wait complete. 2024-09-10T18:50:46.603 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:50:46.604 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:50:46.605 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:46.606 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.606 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.627 DEBUG:teuthology.orchestra.run.smithi107:osd.1> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.1.service 2024-09-10T18:50:46.629 INFO:tasks.cephadm:Deploying osd.2 on smithi107 with /dev/vg_nvme/lv_2... 2024-09-10T18:50:46.629 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: pgmap v51: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-09-10T18:50:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:50:46.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:46.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:47.388 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:47.599 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[29404]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[29404]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[29404]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[40355]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[40355]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:47.600 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-mon[40355]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:47.600 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T18:50:47.306+0000 7f7f8b36f640 -1 osd.1 0 waiting for initial osdmap 2024-09-10T18:50:47.601 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:50:47 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T18:50:47.311+0000 7f7f851a4640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:50:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:47 smithi162 ceph-mon[33228]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-10T18:50:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:47 smithi162 ceph-mon[33228]: osdmap e11: 2 total, 1 up, 2 in 2024-09-10T18:50:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:47 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:47 smithi162 ceph-mon[33228]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:50:48.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[29404]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[29404]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[29404]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[40355]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[40355]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[40355]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:48.572 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:48 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:48 smithi162 ceph-mon[33228]: pgmap v53: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:48 smithi162 ceph-mon[33228]: from='osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353]' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:50:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:48 smithi162 ceph-mon[33228]: osdmap e12: 2 total, 1 up, 2 in 2024-09-10T18:50:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:48 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:48 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.570 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] boot 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] boot 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:50:49.571 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:49 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: osd.1 [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] boot 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: osdmap e13: 2 total, 2 up, 2 in 2024-09-10T18:50:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:49 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:50:50.045 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:50:50.129 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:50:50.144 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-10T18:50:50.144 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-10T18:50:50.145 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129584 s, 809 MB/s 2024-09-10T18:50:50.145 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-10T18:50:50.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:50 smithi107 ceph-mon[29404]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:50.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:50 smithi107 ceph-mon[29404]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:50:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:50 smithi107 ceph-mon[40355]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:50 smithi107 ceph-mon[40355]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:50:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:50 smithi162 ceph-mon[33228]: pgmap v56: 0 pgs: ; 0 B data, 290 MiB used, 89 GiB / 89 GiB avail 2024-09-10T18:50:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:50 smithi162 ceph-mon[33228]: osdmap e14: 2 total, 2 up, 2 in 2024-09-10T18:50:51.360 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_2 2024-09-10T18:50:51.601 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:50:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi107 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[29404]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi107 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:51.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:51 smithi107 ceph-mon[40355]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi107 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 8031M 2024-09-10T18:50:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:50:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:50:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:51 smithi162 ceph-mon[33228]: pgmap v58: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:53.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 -- 172.21.15.107:0/1370743893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 msgr2=0x7fc62c0fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:53.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 --2- 172.21.15.107:0/1370743893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c0fcc60 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fc628004890 tx=0x7fc628009da0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 -- 172.21.15.107:0/1370743893 shutdown_connections 2024-09-10T18:50:53.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 --2- 172.21.15.107:0/1370743893 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 0x7fc62c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.142 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 --2- 172.21.15.107:0/1370743893 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 --2- 172.21.15.107:0/1370743893 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c0fcc60 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 -- 172.21.15.107:0/1370743893 >> 172.21.15.107:0/1370743893 conn(0x7fc62c0f8010 msgr2=0x7fc62c0fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 -- 172.21.15.107:0/1370743893 shutdown_connections 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.139+0000 7fc633863640 1 -- 172.21.15.107:0/1370743893 wait complete. 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 Processor -- start 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 -- start start 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c10d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 0x7fc62c114190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c114760 con 0x7fc62c0fdb20 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c1148d0 con 0x7fc62c0fe4d0 2024-09-10T18:50:53.143 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc633863640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc62c114a90 con 0x7fc62c0fc850 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc6315ff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c10d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc6315ff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:39718/0 (socket says 172.21.15.107:39718) 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc6315ff640 1 -- 172.21.15.107:0/3544042067 learned_addr learned my addr 172.21.15.107:0/3544042067 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc631e00640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 0x7fc62c114190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 -- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 msgr2=0x7fc62c10cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 -- 172.21.15.107:0/3544042067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 msgr2=0x7fc62c114190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 0x7fc62c114190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 -- 172.21.15.107:0/3544042067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc61c005040 con 0x7fc62c0fdb20 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc6315ff640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.140+0000 7fc630dfe640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c10d170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0093e0 tx=0x7fc61c009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:53.144 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.141+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c007ad0 con 0x7fc62c0fdb20 2024-09-10T18:50:53.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.141+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc61c00f040 con 0x7fc62c0fdb20 2024-09-10T18:50:53.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.141+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc61c013ab0 con 0x7fc62c0fdb20 2024-09-10T18:50:53.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.141+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc628005040 con 0x7fc62c0fdb20 2024-09-10T18:50:53.145 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.141+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc62c115000 con 0x7fc62c0fdb20 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.142+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc62c04bf60 con 0x7fc62c0fdb20 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.142+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 14) v1 ==== 90725+0+0 (secure 0 0 0) 0x7fc61c00e930 con 0x7fc62c0fdb20 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.142+0000 7fc621ffb640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 0x7fc618071240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.143+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(14..14 src has 1..14) v4 ==== 2195+0+0 (secure 0 0 0) 0x7fc61c08d310 con 0x7fc62c0fdb20 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.143+0000 7fc6315ff640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 0x7fc618071240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:50:53.146 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.143+0000 7fc6315ff640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 0x7fc618071240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc628004480 tx=0x7fc628009da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:50:53.148 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.147+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc61c057420 con 0x7fc62c0fdb20 2024-09-10T18:50:53.314 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:50:53.313+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fc62c107620 con 0x7fc61806ed90 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[29404]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[29404]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[40355]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[40355]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:54.392 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:54.393 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:54 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:54 smithi162 ceph-mon[33228]: pgmap v59: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:54 smithi162 ceph-mon[33228]: from='client.14262 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:50:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:54 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:50:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:54 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:50:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:54 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:50:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2369836461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]': finished 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1286246111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2369836461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]': finished 2024-09-10T18:50:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:50:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:50:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1286246111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: pgmap v60: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2369836461' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]: dispatch 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c61bf972-04c3-4f95-b23f-89662ed8c66d"}]': finished 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: osdmap e15: 3 total, 2 up, 3 in 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:50:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1286246111' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:50:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:50:58 smithi107 ceph-mon[29404]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:50:58 smithi107 ceph-mon[40355]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:50:58.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:50:58 smithi162 ceph-mon[33228]: pgmap v62: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:00.397 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:00 smithi107 ceph-mon[29404]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:00.397 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:00 smithi107 ceph-mon[40355]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:00 smithi162 ceph-mon[33228]: pgmap v63: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[29404]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[29404]: Deploying daemon osd.2 on smithi107 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[40355]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:02.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:51:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:02 smithi107 ceph-mon[40355]: Deploying daemon osd.2 on smithi107 2024-09-10T18:51:02.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:02 smithi162 ceph-mon[33228]: pgmap v64: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-09-10T18:51:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:02 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:02 smithi162 ceph-mon[33228]: Deploying daemon osd.2 on smithi107 2024-09-10T18:51:04.232 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:04 smithi107 ceph-mon[29404]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:04.233 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:04 smithi107 ceph-mon[40355]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:04 smithi162 ceph-mon[33228]: pgmap v65: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:06.360 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:06 smithi107 ceph-mon[29404]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:06.361 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:06 smithi107 ceph-mon[40355]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:06 smithi162 ceph-mon[33228]: pgmap v66: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:07.337 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:07 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:07 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:07 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:07 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:08 smithi162 ceph-mon[33228]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:08 smithi107 ceph-mon[29404]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:08 smithi107 ceph-mon[40355]: pgmap v67: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:09.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.378+0000 7fc621ffb640 1 -- 172.21.15.107:0/3544042067 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fc62c107620 con 0x7fc61806ed90 2024-09-10T18:51:09.380 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 2 on host 'smithi107' 2024-09-10T18:51:09.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 msgr2=0x7fc618071240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 0x7fc618071240 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fc628004480 tx=0x7fc628009da0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 msgr2=0x7fc62c10d170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c10d170 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fc61c0093e0 tx=0x7fc61c009da0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 shutdown_connections 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc61806ed90 0x7fc618071240 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc62c0fe4d0 0x7fc62c114190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc62c0fdb20 0x7fc62c10d170 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 --2- 172.21.15.107:0/3544042067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc62c0fc850 0x7fc62c10cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 >> 172.21.15.107:0/3544042067 conn(0x7fc62c0f8010 msgr2=0x7fc62c0f9930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 shutdown_connections 2024-09-10T18:51:09.382 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:09.380+0000 7fc633863640 1 -- 172.21.15.107:0/3544042067 wait complete. 2024-09-10T18:51:10.109 DEBUG:teuthology.orchestra.run.smithi107:osd.2> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.2.service 2024-09-10T18:51:10.111 INFO:tasks.cephadm:Deploying osd.3 on smithi107 with /dev/vg_nvme/lv_1... 2024-09-10T18:51:10.112 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:51:10.503 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[29404]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[29404]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[40355]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.646 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.647 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[40355]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:10.647 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:10 smithi107 ceph-mon[40355]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:10 smithi162 ceph-mon[33228]: pgmap v68: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:10 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:10 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:10 smithi162 ceph-mon[33228]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:10 smithi162 ceph-mon[33228]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-09-10T18:51:11.922 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[40355]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:51:11.922 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[40355]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:51:11.922 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:11.923 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[40355]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:11.923 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[40355]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:11.923 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:51:11.923 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[29404]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:51:11.924 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:11.924 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[29404]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:11.924 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:11.925 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2[57503]: 2024-09-10T18:51:11.628+0000 7f8dbbb19640 -1 osd.2 0 waiting for initial osdmap 2024-09-10T18:51:11.925 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:11 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2[57503]: 2024-09-10T18:51:11.632+0000 7f8db7952640 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:51:11.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:11 smithi162 ceph-mon[33228]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-10T18:51:11.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:11 smithi162 ceph-mon[33228]: osdmap e16: 3 total, 2 up, 3 in 2024-09-10T18:51:11.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:11 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:11.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:11 smithi162 ceph-mon[33228]: from='osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:11.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:11 smithi162 ceph-mon[33228]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:12.281 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:51:12.370 INFO:teuthology.orchestra.run.smithi107.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:51:12.386 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10+0 records in 2024-09-10T18:51:12.386 INFO:teuthology.orchestra.run.smithi107.stderr:10+0 records out 2024-09-10T18:51:12.386 INFO:teuthology.orchestra.run.smithi107.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0132938 s, 789 MB/s 2024-09-10T18:51:12.386 INFO:teuthology.orchestra.run.smithi107.stderr:--> Zapping successful for: 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:12.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: pgmap v70: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:12.935 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:12.936 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: osdmap e17: 3 total, 2 up, 3 in 2024-09-10T18:51:12.936 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.936 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:12.936 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:12.936 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:13.267 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi107:vg_nvme/lv_1 2024-09-10T18:51:13.587 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] boot 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:13 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] boot 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:51:13.956 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: osd.2 [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] boot 2024-09-10T18:51:13.957 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: osdmap e18: 3 total, 3 up, 3 in 2024-09-10T18:51:13.957 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:51:13.957 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:13 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]: dispatch 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[29404]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[29404]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[40355]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[40355]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:51:14.862 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:14 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:14.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:14 smithi162 ceph-mon[33228]: pgmap v73: 0 pgs: ; 0 B data, 581 MiB used, 178 GiB / 179 GiB avail 2024-09-10T18:51:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:14 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32}]': finished 2024-09-10T18:51:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:14 smithi162 ceph-mon[33228]: osdmap e19: 3 total, 3 up, 3 in 2024-09-10T18:51:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:14 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:51:15.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61371]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:15.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61371]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:15.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61371]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:15.466 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61371]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:15.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:15.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[29404]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:51:15.745 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:15.745 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:15.745 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[40355]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:51:15.746 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:15 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:15.746 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61788]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:15.746 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61788]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:15.746 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61788]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:15.746 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61788]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:15 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:51:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:15 smithi162 ceph-mon[33228]: osdmap e20: 3 total, 3 up, 3 in 2024-09-10T18:51:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:15 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:16.001 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61809]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:16.001 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61809]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:16.001 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61809]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:16.001 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:15 smithi107 sudo[61809]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:16.260 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:16.260 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:16.260 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61814]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:16.260 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61814]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:16.516 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61843]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:16.516 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61843]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:16.516 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61843]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:16.516 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61843]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:16.778 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61849]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:16.778 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61849]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:16.778 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61849]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:16.779 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61849]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:16.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 -- 172.21.15.107:0/1070535224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fd3e0 msgr2=0x7fe094103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:16.831 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 --2- 172.21.15.107:0/1070535224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fd3e0 0x7fe094103c80 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fe08c009cd0 tx=0x7fe08c004d10 comp rx=0 tx=0).stop 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 -- 172.21.15.107:0/1070535224 shutdown_connections 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 --2- 172.21.15.107:0/1070535224 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fd3e0 0x7fe094103c80 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 --2- 172.21.15.107:0/1070535224 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fca30 0x7fe0940fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 --2- 172.21.15.107:0/1070535224 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fb760 0x7fe0940fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 -- 172.21.15.107:0/1070535224 >> 172.21.15.107:0/1070535224 conn(0x7fe0940f6f20 msgr2=0x7fe0940f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 -- 172.21.15.107:0/1070535224 shutdown_connections 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.805+0000 7fe099fe1640 1 -- 172.21.15.107:0/1070535224 wait complete. 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 Processor -- start 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 -- start start 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fca30 0x7fe09410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:16.832 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fd3e0 0x7fe094113eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe094114480 con 0x7fe0940fd3e0 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0941145f0 con 0x7fe0940fca30 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe099fe1640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0941147b0 con 0x7fe0940fb760 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe093fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fd3e0 0x7fe094113eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe0937fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:42372/0 (socket says 172.21.15.107:42372) 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe0937fe640 1 -- 172.21.15.107:0/109960127 learned_addr learned my addr 172.21.15.107:0/109960127 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe092ffd640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fca30 0x7fe09410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.806+0000 7fe0937fe640 1 -- 172.21.15.107:0/109960127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fca30 msgr2=0x7fe09410ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe0937fe640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fca30 0x7fe09410ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe0937fe640 1 -- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fd3e0 msgr2=0x7fe094113eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe0937fe640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fd3e0 0x7fe094113eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:16.833 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe0937fe640 1 -- 172.21.15.107:0/109960127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c0098b0 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe0937fe640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe084002b00 tx=0x7fe084004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084007c30 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fe08400f040 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0940fdcb0 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe084009340 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.807+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0940fe180 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.808+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe09404bf60 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.809+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fe084009730 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.809+0000 7fe07bfff640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 0x7fe07c071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.809+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 3013+0+0 (secure 0 0 0) 0x7fe08408d530 con 0x7fe0940fb760 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.809+0000 7fe092ffd640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 0x7fe07c071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.810+0000 7fe092ffd640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 0x7fe07c071300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09410de10 tx=0x7fe08800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:51:16.834 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.814+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe084057170 con 0x7fe0940fb760 2024-09-10T18:51:16.985 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:16.982+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe0940f9040 con 0x7fe07c06ee50 2024-09-10T18:51:17.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61859]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:17.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61859]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:17.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61859]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:17.042 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:16 smithi107 sudo[61859]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:17.308 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 ceph-mon[33228]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:17.308 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 ceph-mon[33228]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 ceph-mon[33228]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35899]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35899]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35899]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:17.309 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35899]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:17.458 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[40355]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[40355]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[40355]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[61882]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[61882]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[61882]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[61882]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:17.485 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[29404]: pgmap v76: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:17.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[29404]: osdmap e21: 3 total, 3 up, 3 in 2024-09-10T18:51:17.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[29404]: mgrmap e15: y(active, since 2m), standbys: x 2024-09-10T18:51:17.486 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:17 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35902]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35902]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35902]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:17 smithi162 sudo[35902]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:18.085 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[62076]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-10T18:51:18.085 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[62076]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:18.085 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[62076]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:18.085 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:17 smithi107 sudo[62076]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi107 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.430 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:18 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:18.446 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 sudo[62082]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 sudo[62082]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 sudo[62082]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 sudo[62082]: pam_unix(sudo:session): session closed for user root 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi107 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:18.447 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-10T18:51:18.448 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.449 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi107 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi107:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:18.450 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: pgmap v78: 1 pgs: 1 creating+peering; 0 B data, 872 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 5354M 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:51:18.451 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:18.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:51:18.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:51:18.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:18.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:18.452 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:18 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:19.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:19.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[29404]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:19.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:19.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:19.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[40355]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:19.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:19 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-10T18:51:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:19 smithi162 ceph-mon[33228]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-10T18:51:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:20 smithi162 ceph-mon[33228]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]: dispatch 2024-09-10T18:51:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]': finished 2024-09-10T18:51:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:20 smithi162 ceph-mon[33228]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:51:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[29404]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]: dispatch 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]': finished 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[29404]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[40355]: pgmap v79: 1 pgs: 1 creating+peering; 0 B data, 873 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]: dispatch 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1132565961' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "b42bf6e2-ce5b-446e-82e7-ea673401b1bb"}]': finished 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[40355]: osdmap e22: 4 total, 3 up, 4 in 2024-09-10T18:51:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:21.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1894276167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:21.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1894276167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:21.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1894276167' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:22.368 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:22 smithi107 ceph-mon[29404]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:22.368 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:22 smithi107 ceph-mon[40355]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:22 smithi162 ceph-mon[33228]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:24.418 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:24 smithi107 ceph-mon[29404]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:24.419 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:24 smithi107 ceph-mon[40355]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:24 smithi162 ceph-mon[33228]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[29404]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[29404]: Deploying daemon osd.3 on smithi107 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[40355]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:26.354 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:26 smithi107 ceph-mon[40355]: Deploying daemon osd.3 on smithi107 2024-09-10T18:51:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:26 smithi162 ceph-mon[33228]: pgmap v83: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:26 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-09-10T18:51:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:26 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:26 smithi162 ceph-mon[33228]: Deploying daemon osd.3 on smithi107 2024-09-10T18:51:28.457 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:28 smithi107 ceph-mon[29404]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:28.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:28 smithi107 ceph-mon[40355]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:28 smithi162 ceph-mon[33228]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:30.331 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:30 smithi107 ceph-mon[29404]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:30.332 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:30 smithi107 ceph-mon[40355]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:30.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:30 smithi162 ceph-mon[33228]: pgmap v85: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:31.251 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:31.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:31.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:31.252 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:31.252 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:31.252 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:31 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:31 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:31 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:31 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:32 smithi107 ceph-mon[29404]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:32 smithi107 ceph-mon[40355]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:32.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:32 smithi162 ceph-mon[33228]: pgmap v86: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:33.992 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.990+0000 7fe07bfff640 1 -- 172.21.15.107:0/109960127 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe0940f9040 con 0x7fe07c06ee50 2024-09-10T18:51:33.992 INFO:teuthology.orchestra.run.smithi107.stdout:Created osd(s) 3 on host 'smithi107' 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 msgr2=0x7fe07c071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 0x7fe07c071300 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe09410de10 tx=0x7fe08800b040 comp rx=0 tx=0).stop 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 msgr2=0x7fe09410c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fe084002b00 tx=0x7fe084004600 comp rx=0 tx=0).stop 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 shutdown_connections 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe07c06ee50 0x7fe07c071300 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe0940fd3e0 0x7fe094113eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:33.995 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe0940fca30 0x7fe09410ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:33.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 --2- 172.21.15.107:0/109960127 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe0940fb760 0x7fe09410c910 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:33.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 >> 172.21.15.107:0/109960127 conn(0x7fe0940f6f20 msgr2=0x7fe0940f8930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:51:33.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.993+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 shutdown_connections 2024-09-10T18:51:33.996 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:51:33.994+0000 7fe099fe1640 1 -- 172.21.15.107:0/109960127 wait complete. 2024-09-10T18:51:34.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:34.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:34 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.600 DEBUG:teuthology.orchestra.run.smithi107:osd.3> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.3.service 2024-09-10T18:51:34.603 INFO:tasks.cephadm:Deploying osd.4 on smithi162 with /dev/vg_nvme/lv_4... 2024-09-10T18:51:34.603 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-09-10T18:51:34.632 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:34.633 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.633 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:34 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:34.851 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:51:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:35 smithi162 ceph-mon[33228]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:51:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:35 smithi162 ceph-mon[33228]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:51:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:35 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:35 smithi162 ceph-mon[33228]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:35 smithi162 ceph-mon[33228]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[29404]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[29404]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[29404]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[29404]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.925 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[40355]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-10T18:51:35.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[40355]: osdmap e23: 4 total, 3 up, 4 in 2024-09-10T18:51:35.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:35.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[40355]: from='osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.926 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-mon[40355]: from='osd.3 ' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]: dispatch 2024-09-10T18:51:35.926 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T18:51:35.634+0000 7fc9dfaf2640 -1 osd.3 0 waiting for initial osdmap 2024-09-10T18:51:35.926 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 18:51:35 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T18:51:35.638+0000 7fc9da929640 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:51:36.665 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-09-10T18:51:36.754 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-09-10T18:51:36.768 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-09-10T18:51:36.768 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-09-10T18:51:36.768 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114917 s, 912 MB/s 2024-09-10T18:51:36.768 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-09-10T18:51:36.807 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[29404]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[29404]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[29404]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[40355]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[40355]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[40355]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:36.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:36 smithi162 ceph-mon[33228]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 874 MiB used, 267 GiB / 268 GiB avail 2024-09-10T18:51:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:36 smithi162 ceph-mon[33228]: from='osd.3 ' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi107", "root=default"]}]': finished 2024-09-10T18:51:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:36 smithi162 ceph-mon[33228]: osdmap e24: 4 total, 3 up, 4 in 2024-09-10T18:51:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:37.389 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_4 2024-09-10T18:51:37.684 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:51:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:51:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:51:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] boot 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] boot 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:37.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:37.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:37 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.805 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:51:37.805 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:51:37.805 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: osd.3 [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] boot 2024-09-10T18:51:37.805 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: osdmap e25: 4 total, 4 up, 4 in 2024-09-10T18:51:37.805 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:51:37.806 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:37.807 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:37.807 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:37.807 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:37 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:38 smithi162 ceph-mon[33228]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:38 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi107 2024-09-10T18:51:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:38 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-10T18:51:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:38 smithi162 ceph-mon[33228]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[29404]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi107 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[29404]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[40355]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:38.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi107 2024-09-10T18:51:38.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 4015M 2024-09-10T18:51:38.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:38 smithi107 ceph-mon[40355]: osdmap e26: 4 total, 4 up, 4 in 2024-09-10T18:51:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 -- 172.21.15.162:0/2768295767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fb760 msgr2=0x7fb0340fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/2768295767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fb760 0x7fb0340fbb70 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fb030004950 tx=0x7fb030009da0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 -- 172.21.15.162:0/2768295767 shutdown_connections 2024-09-10T18:51:39.180 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/2768295767 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fd3e0 0x7fb034103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/2768295767 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fca30 0x7fb0340fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/2768295767 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fb760 0x7fb0340fbb70 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 -- 172.21.15.162:0/2768295767 >> 172.21.15.162:0/2768295767 conn(0x7fb0340f6f20 msgr2=0x7fb0340f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:51:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.151+0000 7fb03c7c7640 1 -- 172.21.15.162:0/2768295767 shutdown_connections 2024-09-10T18:51:39.181 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 -- 172.21.15.162:0/2768295767 wait complete. 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 Processor -- start 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 -- start start 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fb760 0x7fb03410ccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:39.182 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fd3e0 0x7fb034114260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034114830 con 0x7fb0340fb760 2024-09-10T18:51:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0341149a0 con 0x7fb0340fca30 2024-09-10T18:51:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03c7c7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb034114b60 con 0x7fb0340fd3e0 2024-09-10T18:51:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb039d62640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:39.183 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb039d62640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34012/0 (socket says 172.21.15.162:34012) 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb039d62640 1 -- 172.21.15.162:0/3834477361 learned_addr learned my addr 172.21.15.162:0/3834477361 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03a563640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fb760 0x7fb03410ccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.152+0000 7fb03ad64640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fd3e0 0x7fb034114260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 -- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fd3e0 msgr2=0x7fb034114260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fd3e0 0x7fb034114260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.184 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 -- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fb760 msgr2=0x7fb03410ccd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fb760 0x7fb03410ccd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 -- 172.21.15.162:0/3834477361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030005040 con 0x7fb0340fca30 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb039d62640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b810 tx=0x7fb0240091d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02400be60 con 0x7fb0340fca30 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fb02400cb70 con 0x7fb0340fca30 2024-09-10T18:51:39.185 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034114e40 con 0x7fb0340fca30 2024-09-10T18:51:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0240159f0 con 0x7fb0340fca30 2024-09-10T18:51:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.153+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb034115230 con 0x7fb0340fca30 2024-09-10T18:51:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.154+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03404bf60 con 0x7fb0340fca30 2024-09-10T18:51:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.155+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fb024012070 con 0x7fb0340fca30 2024-09-10T18:51:39.186 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.155+0000 7fb02affd640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 0x7fb020071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:51:39.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.155+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 3445+0+0 (secure 0 0 0) 0x7fb02408cf70 con 0x7fb0340fca30 2024-09-10T18:51:39.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.155+0000 7fb03a563640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 0x7fb020071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:51:39.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.156+0000 7fb03a563640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 0x7fb020071300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb030009040 tx=0x7fb030035040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:51:39.187 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.159+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb024056ba0 con 0x7fb0340fca30 2024-09-10T18:51:39.321 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:39.319+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) v1 -- 0x7fb0341076e0 con 0x7fb02006ee50 2024-09-10T18:51:39.904 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:39 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:39.904 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:39 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:39.904 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:39 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:51:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:51:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:39 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:40 smithi162 ceph-mon[33228]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:40 smithi162 ceph-mon[33228]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:40 smithi107 ceph-mon[29404]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:40 smithi107 ceph-mon[29404]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:40.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:40 smithi107 ceph-mon[40355]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:40.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:40 smithi107 ceph-mon[40355]: from='client.24169 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:51:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/547976502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]': finished 2024-09-10T18:51:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:41 smithi162 ceph-mon[33228]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:51:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:41 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/547976502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]': finished 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[29404]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/547976502' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]: dispatch 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6de68f17-c6b5-4b80-8a69-e27860c86651"}]': finished 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[40355]: osdmap e27: 5 total, 4 up, 5 in 2024-09-10T18:51:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:41 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:42 smithi162 ceph-mon[33228]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/4128821087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:42 smithi107 ceph-mon[29404]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/4128821087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:42 smithi107 ceph-mon[40355]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/4128821087' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:51:44.851 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:44 smithi162 ceph-mon[33228]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:44 smithi107 ceph-mon[29404]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:44 smithi107 ceph-mon[40355]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:46.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:46 smithi162 ceph-mon[33228]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:46 smithi107 ceph-mon[29404]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:46.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:46 smithi107 ceph-mon[40355]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:47 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:51:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:47 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:47 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:51:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:47 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:47 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-09-10T18:51:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:47 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:48.802 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:48 smithi162 ceph-mon[33228]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:48.803 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:48 smithi162 ceph-mon[33228]: Deploying daemon osd.4 on smithi162 2024-09-10T18:51:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:48 smithi107 ceph-mon[29404]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:48 smithi107 ceph-mon[29404]: Deploying daemon osd.4 on smithi162 2024-09-10T18:51:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:48 smithi107 ceph-mon[40355]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:48 smithi107 ceph-mon[40355]: Deploying daemon osd.4 on smithi162 2024-09-10T18:51:50.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:50 smithi162 ceph-mon[33228]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:50 smithi107 ceph-mon[29404]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:50 smithi107 ceph-mon[40355]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:52.940 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:52 smithi162 ceph-mon[33228]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:52.940 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:52.940 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:52.940 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[29404]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[40355]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:54.962 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:54 smithi162 ceph-mon[33228]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:54 smithi107 ceph-mon[29404]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:54 smithi107 ceph-mon[40355]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:55.689 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.686+0000 7fb02affd640 1 -- 172.21.15.162:0/3834477361 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fb0341076e0 con 0x7fb02006ee50 2024-09-10T18:51:55.692 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 4 on host 'smithi162' 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 msgr2=0x7fb020071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 0x7fb020071300 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fb030009040 tx=0x7fb030035040 comp rx=0 tx=0).stop 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 msgr2=0x7fb03410d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb02400b810 tx=0x7fb0240091d0 comp rx=0 tx=0).stop 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 shutdown_connections 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fb02006ee50 0x7fb020071300 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb0340fd3e0 0x7fb034114260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:55.693 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb0340fca30 0x7fb03410d210 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:55.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 --2- 172.21.15.162:0/3834477361 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb0340fb760 0x7fb03410ccd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:51:55.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 >> 172.21.15.162:0/3834477361 conn(0x7fb0340f6f20 msgr2=0x7fb0340f8880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:51:55.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 shutdown_connections 2024-09-10T18:51:55.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:51:55.689+0000 7fb03c7c7640 1 -- 172.21.15.162:0/3834477361 wait complete. 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:56.297 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:56.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.298 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.320 DEBUG:teuthology.orchestra.run.smithi162:osd.4> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.4.service 2024-09-10T18:51:56.323 INFO:tasks.cephadm:Deploying osd.5 on smithi162 with /dev/vg_nvme/lv_3... 2024-09-10T18:51:56.323 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:51:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-09-10T18:51:56.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:51:56.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:56.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:51:57.071 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:51:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-mon[33228]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:51:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-mon[33228]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:51:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-mon[33228]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-mon[33228]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:57.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T18:51:57.283+0000 7f177b714640 -1 osd.4 0 waiting for initial osdmap 2024-09-10T18:51:57.675 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 18:51:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T18:51:57.288+0000 7f1777d4e640 -1 osd.4 29 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:51:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[29404]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[29404]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[40355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-10T18:51:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[40355]: osdmap e28: 5 total, 4 up, 5 in 2024-09-10T18:51:57.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:57.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[40355]: from='osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:57.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:57 smithi107 ceph-mon[40355]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:51:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:58 smithi162 ceph-mon[33228]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:58 smithi162 ceph-mon[33228]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:51:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:58 smithi162 ceph-mon[33228]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:51:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:58 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:58 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[29404]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[29404]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:51:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[29404]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:51:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[40355]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 1.1 GiB used, 356 GiB / 358 GiB avail 2024-09-10T18:51:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[40355]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:51:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[40355]: osdmap e29: 5 total, 4 up, 5 in 2024-09-10T18:51:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:58 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] boot 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:59.365 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:51:59 smithi162 ceph-mon[33228]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:51:59.602 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-09-10T18:51:59.694 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-09-10T18:51:59.709 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-09-10T18:51:59.709 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-09-10T18:51:59.709 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0127917 s, 820 MB/s 2024-09-10T18:51:59.710 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-09-10T18:51:59.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:51:59.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] boot 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[29404]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: osd.4 [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] boot 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: osdmap e30: 5 total, 5 up, 5 in 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:51:59.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:51:59 smithi107 ceph-mon[40355]: osdmap e31: 5 total, 5 up, 5 in 2024-09-10T18:52:00.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:00.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.413 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:00.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:00.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:00.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.414 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:00 smithi162 ceph-mon[33228]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:52:00.696 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_3 2024-09-10T18:52:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[29404]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:00 smithi107 ceph-mon[40355]: osdmap e32: 5 total, 5 up, 5 in 2024-09-10T18:52:00.933 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:52:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:01 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi162 2024-09-10T18:52:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:01 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T18:52:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:01 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi162 2024-09-10T18:52:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:01 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T18:52:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:01 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi162 2024-09-10T18:52:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:01 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 17088M 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.508+0000 7fdf139c2640 1 -- 172.21.15.162:0/2891042086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 msgr2=0x7fdf0c104970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.508+0000 7fdf139c2640 1 --2- 172.21.15.162:0/2891042086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c104970 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fdf04009cd0 tx=0x7fdf04004d10 comp rx=0 tx=0).stop 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- 172.21.15.162:0/2891042086 shutdown_connections 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- 172.21.15.162:0/2891042086 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c104970 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- 172.21.15.162:0/2891042086 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c0ff9b0 0x7fdf0c101d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- 172.21.15.162:0/2891042086 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c065f10 0x7fdf0c0ff470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- 172.21.15.162:0/2891042086 >> 172.21.15.162:0/2891042086 conn(0x7fdf0c0f6f20 msgr2=0x7fdf0c0f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- 172.21.15.162:0/2891042086 shutdown_connections 2024-09-10T18:52:02.522 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- 172.21.15.162:0/2891042086 wait complete. 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 Processor -- start 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- start start 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c065f10 0x7fdf0c10cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0ff9b0 0x7fdf0c10d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1146c0 con 0x7fdf0c0ff9b0 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c114830 con 0x7fdf0c1024c0 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf139c2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf0c1149f0 con 0x7fdf0c065f10 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf11f5f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf11f5f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:50198/0 (socket says 172.21.15.162:50198) 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf11f5f640 1 -- 172.21.15.162:0/1250251470 learned_addr learned my addr 172.21.15.162:0/1250251470 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf1175e640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c065f10 0x7fdf0c10cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:02.523 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.509+0000 7fdf10f5d640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0ff9b0 0x7fdf0c10d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 -- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c065f10 msgr2=0x7fdf0c10cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c065f10 0x7fdf0c10cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 -- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0ff9b0 msgr2=0x7fdf0c10d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0ff9b0 0x7fdf0c10d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 -- 172.21.15.162:0/1250251470 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf040098b0 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf11f5f640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402a800 tx=0x7fdf04004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf04038070 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fdf04009250 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf0403cbd0 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf0c114c70 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.510+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf0c1150f0 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.511+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf0c04bf60 con 0x7fdf0c1024c0 2024-09-10T18:52:02.524 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.512+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fdf04030070 con 0x7fdf0c1024c0 2024-09-10T18:52:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.512+0000 7fdf01ffb640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 0x7fdef8071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.513+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 3987+0+0 (secure 0 0 0) 0x7fdf040ae6a0 con 0x7fdf0c1024c0 2024-09-10T18:52:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.513+0000 7fdf1175e640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 0x7fdef8071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.513+0000 7fdf1175e640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 0x7fdef8071300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c065c70 tx=0x7fdf08009120 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:02.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.517+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fdf04077fc0 con 0x7fdf0c1024c0 2024-09-10T18:52:02.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:02 smithi162 ceph-mon[33228]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:02.683 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:02.681+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) v1 -- 0x7fdf0c107700 con 0x7fdef806ee50 2024-09-10T18:52:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:02 smithi107 ceph-mon[29404]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:02 smithi107 ceph-mon[40355]: pgmap v111: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:03.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:03 smithi162 ceph-mon[33228]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:03 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[29404]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[40355]: from='client.24196 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:03 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:04.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:04 smithi107 ceph-mon[29404]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:04.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:04 smithi107 ceph-mon[40355]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:04 smithi162 ceph-mon[33228]: pgmap v112: 1 pgs: 1 peering; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:05.540 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/2245494064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.540 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.540 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]': finished 2024-09-10T18:52:05.540 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:05 smithi162 ceph-mon[33228]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:52:05.540 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/2245494064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]': finished 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[29404]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/2245494064' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]: dispatch 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "88d7424d-a81a-4695-8f1e-cda6b4bc228c"}]': finished 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[40355]: osdmap e33: 6 total, 5 up, 6 in 2024-09-10T18:52:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:06 smithi162 ceph-mon[33228]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:52:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/4062547216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:06 smithi107 ceph-mon[29404]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:52:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/4062547216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:06 smithi107 ceph-mon[40355]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-10T18:52:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/4062547216' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:08.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:08 smithi162 ceph-mon[33228]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:08.730 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:08 smithi107 ceph-mon[29404]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:08.730 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:08 smithi107 ceph-mon[40355]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-10T18:52:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:10 smithi162 ceph-mon[33228]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T18:52:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:10 smithi107 ceph-mon[29404]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T18:52:10.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:10 smithi107 ceph-mon[40355]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-09-10T18:52:11.587 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:11 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:52:11.587 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:11 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:11 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:52:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:11 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:11 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-09-10T18:52:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:11 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:12.478 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:12 smithi162 ceph-mon[33228]: Deploying daemon osd.5 on smithi162 2024-09-10T18:52:12.478 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:12 smithi162 ceph-mon[33228]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:12 smithi107 ceph-mon[29404]: Deploying daemon osd.5 on smithi162 2024-09-10T18:52:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:12 smithi107 ceph-mon[29404]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:12 smithi107 ceph-mon[40355]: Deploying daemon osd.5 on smithi162 2024-09-10T18:52:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:12 smithi107 ceph-mon[40355]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:14.671 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:14 smithi162 ceph-mon[33228]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:14 smithi107 ceph-mon[29404]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:14 smithi107 ceph-mon[40355]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-10T18:52:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:16 smithi162 ceph-mon[33228]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:16 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:16 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:16 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[29404]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[40355]: pgmap v119: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:16 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:18.395 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:18 smithi162 ceph-mon[33228]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:18 smithi107 ceph-mon[29404]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:18.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:18 smithi107 ceph-mon[40355]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:19.705 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.704+0000 7fdf01ffb640 1 -- 172.21.15.162:0/1250251470 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdf0c107700 con 0x7fdef806ee50 2024-09-10T18:52:19.706 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 5 on host 'smithi162' 2024-09-10T18:52:19.707 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 msgr2=0x7fdef8071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:19.707 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 0x7fdef8071300 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fdf0c065c70 tx=0x7fdf08009120 comp rx=0 tx=0).stop 2024-09-10T18:52:19.707 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 msgr2=0x7fdf0c1140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fdf0402a800 tx=0x7fdf04004d10 comp rx=0 tx=0).stop 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 shutdown_connections 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fdef806ee50 0x7fdef8071300 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf0c1024c0 0x7fdf0c1140f0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf0c0ff9b0 0x7fdf0c10d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 --2- 172.21.15.162:0/1250251470 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf0c065f10 0x7fdf0c10cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 >> 172.21.15.162:0/1250251470 conn(0x7fdf0c0f6f20 msgr2=0x7fdf0c0fd960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 shutdown_connections 2024-09-10T18:52:19.708 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:19.706+0000 7fdf139c2640 1 -- 172.21.15.162:0/1250251470 wait complete. 2024-09-10T18:52:19.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:19 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:19 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:20.312 DEBUG:teuthology.orchestra.run.smithi162:osd.5> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.5.service 2024-09-10T18:52:20.315 INFO:tasks.cephadm:Deploying osd.6 on smithi162 with /dev/vg_nvme/lv_2... 2024-09-10T18:52:20.315 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:20.817 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-mon[33228]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:20.817 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T18:52:20.752+0000 7f6adfa98640 -1 osd.5 0 waiting for initial osdmap 2024-09-10T18:52:20.817 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 18:52:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T18:52:20.757+0000 7f6ad98ab640 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:52:21.130 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:52:21.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:21.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:52:21.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: osdmap e34: 6 total, 5 up, 6 in 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: from='osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:20 smithi107 ceph-mon[40355]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:22.031 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:21 smithi162 ceph-mon[33228]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:22.031 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:21 smithi162 ceph-mon[33228]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:52:22.031 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:21 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.031 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:21 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.032 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:21 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[29404]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[29404]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[40355]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[40355]: osdmap e35: 6 total, 5 up, 6 in 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:21 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] boot 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:22.845 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:22 smithi162 ceph-mon[33228]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:52:23.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] boot 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[29404]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 1.4 GiB used, 446 GiB / 447 GiB avail 2024-09-10T18:52:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: osd.5 [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] boot 2024-09-10T18:52:23.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: osdmap e36: 6 total, 6 up, 6 in 2024-09-10T18:52:23.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:52:23.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:22 smithi107 ceph-mon[40355]: osdmap e37: 6 total, 6 up, 6 in 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.379 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.380 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:24.380 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:24.380 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: osdmap e38: 6 total, 6 up, 6 in 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: pgmap v128: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:24.499 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-09-10T18:52:24.598 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-09-10T18:52:24.612 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-09-10T18:52:24.612 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-09-10T18:52:24.613 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129513 s, 810 MB/s 2024-09-10T18:52:24.613 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-09-10T18:52:25.240 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:25 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi162 2024-09-10T18:52:25.241 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:25 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T18:52:25.266 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_2 2024-09-10T18:52:25.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:25 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi162 2024-09-10T18:52:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:25 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T18:52:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:25 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi162 2024-09-10T18:52:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:25 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 8544M 2024-09-10T18:52:25.506 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:52:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:26 smithi162 ceph-mon[33228]: pgmap v129: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:26 smithi107 ceph-mon[29404]: pgmap v129: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:26 smithi107 ceph-mon[40355]: pgmap v129: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 -- 172.21.15.162:0/3228140999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 msgr2=0x7fe2b80fce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 --2- 172.21.15.162:0/3228140999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b80fce80 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac004950 tx=0x7fe2ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 -- 172.21.15.162:0/3228140999 shutdown_connections 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 --2- 172.21.15.162:0/3228140999 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 0x7fe2b8103c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 --2- 172.21.15.162:0/3228140999 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b80fce80 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 --2- 172.21.15.162:0/3228140999 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 0x7fe2b80fbb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 -- 172.21.15.162:0/3228140999 >> 172.21.15.162:0/3228140999 conn(0x7fe2b80f6f00 msgr2=0x7fe2b80f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 -- 172.21.15.162:0/3228140999 shutdown_connections 2024-09-10T18:52:27.225 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 -- 172.21.15.162:0/3228140999 wait complete. 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.164+0000 7fe2bd562640 1 Processor -- start 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 -- start start 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 0x7fe2b810ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 0x7fe2b81143c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8114990 con 0x7fe2b80fd3c0 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8114b00 con 0x7fe2b80fca10 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2b8114cc0 con 0x7fe2b80fb740 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:48892/0 (socket says 172.21.15.162:48892) 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 -- 172.21.15.162:0/2853432157 learned_addr learned my addr 172.21.15.162:0/2853432157 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b77fe640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 0x7fe2b81143c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:27.226 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b6ffd640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 0x7fe2b810ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 -- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 msgr2=0x7fe2b810ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 0x7fe2b810ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 -- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 msgr2=0x7fe2b81143c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 0x7fe2b81143c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 -- 172.21.15.162:0/2853432157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2ac005040 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2b67fc640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac02d040 tx=0x7fe2ac035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac033420 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fe2ac0339c0 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2ac03b650 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.165+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2b8114f40 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.166+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2b81152d0 con 0x7fe2b80fca10 2024-09-10T18:52:27.227 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.166+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2b804bf60 con 0x7fe2b80fca10 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.167+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fe2ac030020 con 0x7fe2b80fca10 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.167+0000 7fe29f7fe640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 0x7fe2a0071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.167+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 4419+0+0 (secure 0 0 0) 0x7fe2ac0ada80 con 0x7fe2b80fca10 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.168+0000 7fe2b6ffd640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 0x7fe2a0071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.168+0000 7fe2b6ffd640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 0x7fe2a0071300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009b50 tx=0x7fe2a8009120 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:27.228 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.171+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe2ac077330 con 0x7fe2b80fca10 2024-09-10T18:52:27.335 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:27.334+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2b80f8ea0 con 0x7fe2a006ee50 2024-09-10T18:52:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:28 smithi162 ceph-mon[33228]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:28 smithi162 ceph-mon[33228]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:28 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[29404]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[29404]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[40355]: pgmap v130: 1 pgs: 1 peering; 0 B data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[40355]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:28 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/2800865408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]': finished 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:52:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:30 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/2800865408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]': finished 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/2800865408' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]: dispatch 2024-09-10T18:52:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ccaefb3e-2fdf-4af2-a846-6ed2dcae1119"}]': finished 2024-09-10T18:52:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: osdmap e39: 7 total, 6 up, 7 in 2024-09-10T18:52:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:30 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/2707959613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:31.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/2707959613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/2707959613' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:32.422 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:32 smithi162 ceph-mon[33228]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:32 smithi107 ceph-mon[29404]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:32 smithi107 ceph-mon[40355]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:34 smithi162 ceph-mon[33228]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:34.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:34 smithi107 ceph-mon[29404]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:34.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:34 smithi107 ceph-mon[40355]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:36 smithi162 ceph-mon[33228]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:52:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[29404]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:52:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[40355]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-09-10T18:52:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:37.431 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:37 smithi162 ceph-mon[33228]: Deploying daemon osd.6 on smithi162 2024-09-10T18:52:37.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:37 smithi107 ceph-mon[29404]: Deploying daemon osd.6 on smithi162 2024-09-10T18:52:37.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:37 smithi107 ceph-mon[40355]: Deploying daemon osd.6 on smithi162 2024-09-10T18:52:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:38 smithi162 ceph-mon[33228]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:38.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:38 smithi107 ceph-mon[29404]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:38 smithi107 ceph-mon[40355]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:40.252 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:40 smithi162 ceph-mon[33228]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:40.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:40 smithi107 ceph-mon[29404]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:40.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:40 smithi107 ceph-mon[40355]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:41.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:41.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:41.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:41.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:41.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:41 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:41.566 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:41 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:41.566 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:41 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:41.566 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:41 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:42 smithi162 ceph-mon[33228]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:42.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:42 smithi107 ceph-mon[29404]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:42.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:42 smithi107 ceph-mon[40355]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:44.459 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.457+0000 7fe29f7fe640 1 -- 172.21.15.162:0/2853432157 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fe2b80f8ea0 con 0x7fe2a006ee50 2024-09-10T18:52:44.459 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 6 on host 'smithi162' 2024-09-10T18:52:44.461 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 msgr2=0x7fe2a0071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 0x7fe2a0071300 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe2a8009b50 tx=0x7fe2a8009120 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 msgr2=0x7fe2b810d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe2ac02d040 tx=0x7fe2ac035040 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 shutdown_connections 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe2a006ee50 0x7fe2a0071300 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe2b80fd3c0 0x7fe2b81143c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe2b80fca10 0x7fe2b810d360 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.459+0000 7fe2bd562640 1 --2- 172.21.15.162:0/2853432157 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe2b80fb740 0x7fe2b810ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.460+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 >> 172.21.15.162:0/2853432157 conn(0x7fe2b80f6f00 msgr2=0x7fe2b80f8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.460+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 shutdown_connections 2024-09-10T18:52:44.462 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:44.460+0000 7fe2bd562640 1 -- 172.21.15.162:0/2853432157 wait complete. 2024-09-10T18:52:44.573 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:44.573 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:44.574 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:44 smithi162 ceph-mon[33228]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[29404]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: pgmap v139: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:44 smithi107 ceph-mon[40355]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-09-10T18:52:45.148 DEBUG:teuthology.orchestra.run.smithi162:osd.6> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.6.service 2024-09-10T18:52:45.150 INFO:tasks.cephadm:Deploying osd.7 on smithi162 with /dev/vg_nvme/lv_1... 2024-09-10T18:52:45.151 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:45.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-mon[33228]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[40355]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-10T18:52:45.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: osdmap e40: 7 total, 6 up, 7 in 2024-09-10T18:52:45.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:45.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:45 smithi107 ceph-mon[29404]: from='osd.6 ' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:52:45.921 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:52:46.041 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T18:52:45.786+0000 7f007f8c2640 -1 osd.6 0 waiting for initial osdmap 2024-09-10T18:52:46.041 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 18:52:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T18:52:45.792+0000 7f007a6d7640 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:52:46.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:46 smithi162 ceph-mon[33228]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:46 smithi162 ceph-mon[33228]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:46 smithi162 ceph-mon[33228]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:52:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[29404]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[29404]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[29404]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:52:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[40355]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[40355]: from='osd.6 ' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:52:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[40355]: osdmap e41: 7 total, 6 up, 7 in 2024-09-10T18:52:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:48.083 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:52:48.083 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:52:48.083 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] boot 2024-09-10T18:52:48.083 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:52:48.084 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:48.084 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:52:48.084 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:47 smithi162 ceph-mon[33228]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:48.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] boot 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[29404]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:52:48.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: osd.6 [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] boot 2024-09-10T18:52:48.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: osdmap e42: 7 total, 7 up, 7 in 2024-09-10T18:52:48.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:52:48.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: osdmap e43: 7 total, 7 up, 7 in 2024-09-10T18:52:48.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:47 smithi107 ceph-mon[40355]: pgmap v145: 1 pgs: 1 peering; 577 KiB data, 1.7 GiB used, 535 GiB / 536 GiB avail 2024-09-10T18:52:48.716 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-09-10T18:52:48.801 INFO:teuthology.orchestra.run.smithi162.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-09-10T18:52:48.817 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10+0 records in 2024-09-10T18:52:48.817 INFO:teuthology.orchestra.run.smithi162.stderr:10+0 records out 2024-09-10T18:52:48.817 INFO:teuthology.orchestra.run.smithi162.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0145451 s, 721 MB/s 2024-09-10T18:52:48.818 INFO:teuthology.orchestra.run.smithi162.stderr:--> Zapping successful for: 2024-09-10T18:52:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:48 smithi162 ceph-mon[33228]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:48 smithi162 ceph-mon[33228]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:52:49.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:48 smithi107 ceph-mon[29404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:49.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:48 smithi107 ceph-mon[29404]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:52:49.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:48 smithi107 ceph-mon[40355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:52:49.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:48 smithi107 ceph-mon[40355]: osdmap e44: 7 total, 7 up, 7 in 2024-09-10T18:52:49.583 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch daemon add osd smithi162:vg_nvme/lv_1 2024-09-10T18:52:49.823 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:52:50.203 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi162 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.204 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:50 smithi162 ceph-mon[33228]: pgmap v147: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:50.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi162 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[29404]: pgmap v147: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi162 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 5696M 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:52:50.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:50 smithi107 ceph-mon[40355]: pgmap v147: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:51.307 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:51 smithi162 ceph-mon[33228]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:51.307 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:51 smithi162 ceph-mon[33228]: Cluster is now healthy 2024-09-10T18:52:51.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:51 smithi107 ceph-mon[29404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:51 smithi107 ceph-mon[29404]: Cluster is now healthy 2024-09-10T18:52:51.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:51 smithi107 ceph-mon[40355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:52:51.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:51 smithi107 ceph-mon[40355]: Cluster is now healthy 2024-09-10T18:52:51.525 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.518+0000 7f3fb022a640 1 -- 172.21.15.162:0/4102131214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fb0b0 msgr2=0x7f3fa8100bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.518+0000 7f3fb022a640 1 --2- 172.21.15.162:0/4102131214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8100bc0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f3fa0009cd0 tx=0x7f3fa0004d10 comp rx=0 tx=0).stop 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- 172.21.15.162:0/4102131214 shutdown_connections 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- 172.21.15.162:0/4102131214 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8100bc0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- 172.21.15.162:0/4102131214 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fc520 0x7f3fa80fa950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- 172.21.15.162:0/4102131214 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fbbd0 0x7f3fa80fbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- 172.21.15.162:0/4102131214 >> 172.21.15.162:0/4102131214 conn(0x7f3fa80f66a0 msgr2=0x7f3fa80f8ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- 172.21.15.162:0/4102131214 shutdown_connections 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- 172.21.15.162:0/4102131214 wait complete. 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 Processor -- start 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- start start 2024-09-10T18:52:51.526 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8112f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 0x7f3fa8113450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa811aa80 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa811abf0 con 0x7f3fa80fc520 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fb022a640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fa811adb0 con 0x7f3fa80fbbd0 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fae7c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fae7c7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:51334/0 (socket says 172.21.15.162:51334) 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.519+0000 7f3fae7c7640 1 -- 172.21.15.162:0/1231601061 learned_addr learned my addr 172.21.15.162:0/1231601061 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fad7c5640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 0x7f3fa8113450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8112f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:51.527 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 -- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 msgr2=0x7f3fa8113450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 0x7f3fa8113450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 -- 172.21.15.162:0/1231601061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 msgr2=0x7f3fa811a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 -- 172.21.15.162:0/1231601061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fa00098b0 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fae7c7640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fad7c5640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 0x7f3fa8113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fadfc6640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8112f10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4002820 tx=0x7f3fa4009f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa4002d50 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f3fa400f040 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fa811b090 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fa40090d0 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.520+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fa811b5e0 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.528 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.521+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fa804bf60 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.522+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7f3fa400d6b0 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.522+0000 7f3f9e7fc640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 0x7f3f94071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.523+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 4851+0+0 (secure 0 0 0) 0x7f3fa408dc60 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.523+0000 7f3fad7c5640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 0x7f3f94071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.523+0000 7f3fad7c5640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 0x7f3f94071300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f98004fd0 tx=0x7f3f9800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:52:51.529 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.527+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3fa4057170 con 0x7f3fa80fb0b0 2024-09-10T18:52:51.694 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:52:51.692+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) v1 -- 0x7f3fa810dc50 con 0x7f3f9406ee50 2024-09-10T18:52:52.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:52 smithi162 ceph-mon[33228]: pgmap v148: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:52.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:52 smithi162 ceph-mon[33228]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:52.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:52.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:52.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:52 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[40355]: pgmap v148: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[40355]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[29404]: pgmap v148: 1 pgs: 1 peering; 0 B data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[29404]: from='client.14388 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi162:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-09-10T18:52:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:52 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:52:54.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:54.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/45379040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]': finished 2024-09-10T18:52:54.209 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/45379040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]': finished 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:52:54.317 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:54 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:52:54.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:52:54.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/45379040' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]: dispatch 2024-09-10T18:52:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9af4b3a1-c67b-4900-bf78-7c93e7efe0b0"}]': finished 2024-09-10T18:52:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: osdmap e45: 8 total, 7 up, 8 in 2024-09-10T18:52:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:54 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:52:55.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/1159389749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:55.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/1159389749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:55.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/1159389749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-09-10T18:52:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:56 smithi107 ceph-mon[29404]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:56 smithi107 ceph-mon[40355]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:56.672 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:56 smithi162 ceph-mon[33228]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:52:58 smithi162 ceph-mon[33228]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:52:58 smithi107 ceph-mon[29404]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:52:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:52:58 smithi107 ceph-mon[40355]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:00 smithi162 ceph-mon[33228]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:53:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:00 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[29404]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:53:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[40355]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-09-10T18:53:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:00 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:01.473 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:01 smithi162 ceph-mon[33228]: Deploying daemon osd.7 on smithi162 2024-09-10T18:53:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:01 smithi107 ceph-mon[29404]: Deploying daemon osd.7 on smithi162 2024-09-10T18:53:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:01 smithi107 ceph-mon[40355]: Deploying daemon osd.7 on smithi162 2024-09-10T18:53:02.311 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:02 smithi162 ceph-mon[33228]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:02 smithi107 ceph-mon[29404]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:02 smithi107 ceph-mon[40355]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:04.361 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:04 smithi162 ceph-mon[33228]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:04.518 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:04 smithi107 ceph-mon[29404]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:04.518 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:04 smithi107 ceph-mon[40355]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:05.562 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:05.562 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:05.562 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:05 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:05 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:06.672 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:06 smithi162 ceph-mon[33228]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:06 smithi107 ceph-mon[29404]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:06.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:06 smithi107 ceph-mon[40355]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:08.606 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:08.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:08 smithi162 ceph-mon[33228]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:08.648 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.647+0000 7f3f9e7fc640 1 -- 172.21.15.162:0/1231601061 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f3fa810dc50 con 0x7f3f9406ee50 2024-09-10T18:53:08.648 INFO:teuthology.orchestra.run.smithi162.stdout:Created osd(s) 7 on host 'smithi162' 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 msgr2=0x7f3f94071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 0x7f3f94071300 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f3f98004fd0 tx=0x7f3f9800b040 comp rx=0 tx=0).stop 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 msgr2=0x7f3fa8112f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8112f10 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f3fa4002820 tx=0x7f3fa4009f30 comp rx=0 tx=0).stop 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 shutdown_connections 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f3f9406ee50 0x7f3f94071300 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fa80fc520 0x7f3fa811a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fa80fbbd0 0x7f3fa8113450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:08.650 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 --2- 172.21.15.162:0/1231601061 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fa80fb0b0 0x7f3fa8112f10 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:08.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 >> 172.21.15.162:0/1231601061 conn(0x7f3fa80f66a0 msgr2=0x7f3fa80fcba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:08.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 shutdown_connections 2024-09-10T18:53:08.651 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:08.649+0000 7f3fb022a640 1 -- 172.21.15.162:0/1231601061 wait complete. 2024-09-10T18:53:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:08 smithi107 ceph-mon[40355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.385 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-mon[33228]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.408 DEBUG:teuthology.orchestra.run.smithi162:osd.7> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.7.service 2024-09-10T18:53:09.411 INFO:tasks.cephadm:Waiting for 8 OSDs to come up... 2024-09-10T18:53:09.411 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd stat -f json 2024-09-10T18:53:09.664 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:09.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:09.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-10T18:53:09.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: osdmap e46: 8 total, 7 up, 8 in 2024-09-10T18:53:09.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:09.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:09 smithi107 ceph-mon[40355]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]: dispatch 2024-09-10T18:53:09.964 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T18:53:09.956+0000 7f785ae16640 -1 osd.7 0 waiting for initial osdmap 2024-09-10T18:53:09.964 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 18:53:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T18:53:09.963+0000 7f7857c51640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-10T18:53:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[29404]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[29404]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:53:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[29404]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[40355]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[40355]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[40355]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:10 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:10.616 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:10 smithi162 ceph-mon[33228]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 2.0 GiB used, 624 GiB / 626 GiB avail 2024-09-10T18:53:10.616 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:10 smithi162 ceph-mon[33228]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi162", "root=default"]}]': finished 2024-09-10T18:53:10.616 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:10 smithi162 ceph-mon[33228]: osdmap e47: 8 total, 7 up, 8 in 2024-09-10T18:53:10.617 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:10 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:10.617 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:10 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:11.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 -- 172.21.15.107:0/3376468515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 msgr2=0x7f2f100fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:11.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 --2- 172.21.15.107:0/3376468515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f100fcc60 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f2f08004890 tx=0x7f2f08009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.454 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 -- 172.21.15.107:0/3376468515 shutdown_connections 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 --2- 172.21.15.107:0/3376468515 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fe4d0 0x7f2f10104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 --2- 172.21.15.107:0/3376468515 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fdb20 0x7f2f100fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 --2- 172.21.15.107:0/3376468515 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f100fcc60 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 -- 172.21.15.107:0/3376468515 >> 172.21.15.107:0/3376468515 conn(0x7f2f100f8010 msgr2=0x7f2f100fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 -- 172.21.15.107:0/3376468515 shutdown_connections 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 -- 172.21.15.107:0/3376468515 wait complete. 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.440+0000 7f2f155dd640 1 Processor -- start 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 -- start start 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fdb20 0x7f2f1006fa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 0x7f2f1006ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:11.455 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f100706e0 con 0x7f2f100fe4d0 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10070850 con 0x7f2f100fdb20 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f10116c50 con 0x7f2f100fc850 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:49048/0 (socket says 172.21.15.107:49048) 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 -- 172.21.15.107:0/1769270006 learned_addr learned my addr 172.21.15.107:0/1769270006 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0f7fe640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 0x7f2f1006ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0e7fc640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fdb20 0x7f2f1006fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 -- 172.21.15.107:0/1769270006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fdb20 msgr2=0x7f2f1006fa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fdb20 0x7f2f1006fa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 -- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 msgr2=0x7f2f1006ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 0x7f2f1006ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.456 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 -- 172.21.15.107:0/1769270006 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ef8005040 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0f7fe640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 0x7f2f1006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f0effd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009110 tx=0x7f2f08009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08009860 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.441+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f08005040 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.442+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f2f0803c8c0 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.442+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f08033650 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.442+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f10117150 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.442+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f1004bf60 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.443+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7f2f08030020 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.444+0000 7f2f077fe640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 0x7f2efc0712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.444+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 5316+0+0 (secure 0 0 0) 0x7f2f080ae090 con 0x7f2f100fc850 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.444+0000 7f2f0e7fc640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 0x7f2efc0712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:11.457 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.444+0000 7f2f0e7fc640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 0x7f2efc0712d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007fd0 tx=0x7f2ef800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:11.458 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.447+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2f080775a0 con 0x7f2f100fc850 2024-09-10T18:53:11.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.605+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) v1 -- 0x7f2f10103790 con 0x7f2f100fc850 2024-09-10T18:53:11.606 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.605+0000 7f2f077fe640 1 -- 172.21.15.107:0/1769270006 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v48) v1 ==== 74+0+130 (secure 0 0 0) 0x7f2f0807adb0 con 0x7f2f100fc850 2024-09-10T18:53:11.607 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 msgr2=0x7f2efc0712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 0x7f2efc0712d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2ef8007fd0 tx=0x7f2ef800b040 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 msgr2=0x7f2f10071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f2f08009110 tx=0x7f2f08009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 shutdown_connections 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f2efc06ee20 0x7f2efc0712d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2f100fe4d0 0x7f2f1006ffa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2f100fdb20 0x7f2f1006fa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 --2- 172.21.15.107:0/1769270006 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2f100fc850 0x7f2f10071390 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:11.609 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 >> 172.21.15.107:0/1769270006 conn(0x7f2f100f8010 msgr2=0x7f2f100f9940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:11.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 shutdown_connections 2024-09-10T18:53:11.610 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:11.607+0000 7f2f155dd640 1 -- 172.21.15.107:0/1769270006 wait complete. 2024-09-10T18:53:12.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: purged_snaps scrub starts 2024-09-10T18:53:12.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: purged_snaps scrub ok 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] boot 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1769270006' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: purged_snaps scrub starts 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: purged_snaps scrub ok 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] boot 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:53:12.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1769270006' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:53:12.248 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: purged_snaps scrub starts 2024-09-10T18:53:12.248 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: purged_snaps scrub ok 2024-09-10T18:53:12.249 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: osd.7 [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] boot 2024-09-10T18:53:12.249 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: osdmap e48: 8 total, 8 up, 8 in 2024-09-10T18:53:12.249 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:53:12.249 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:12.249 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1769270006' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-09-10T18:53:12.334 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":48,"num_osds":8,"num_up_osds":8,"osd_up_since":1725994390,"num_in_osds":8,"osd_in_since":1725994373,"num_remapped_pgs":1} 2024-09-10T18:53:12.335 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:53:12.575 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:13.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:53:13.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: Detected new or changed devices on smithi162 2024-09-10T18:53:13.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: Detected new or changed devices on smithi162 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:13.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:12 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: osdmap e49: 8 total, 8 up, 8 in 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: Detected new or changed devices on smithi162 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 4272M 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:12 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:14.018 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:13 smithi107 ceph-mon[29404]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:53:14.019 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:13 smithi107 ceph-mon[29404]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:14.019 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:13 smithi107 ceph-mon[40355]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:53:14.019 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:13 smithi107 ceph-mon[40355]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:14.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 -- 172.21.15.107:0/3162177315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 msgr2=0x7f25b8109630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:14.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 --2- 172.21.15.107:0/3162177315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8109630 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f25b0009cd0 tx=0x7f25b0004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:14.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 -- 172.21.15.107:0/3162177315 shutdown_connections 2024-09-10T18:53:14.218 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 --2- 172.21.15.107:0/3162177315 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8109630 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 --2- 172.21.15.107:0/3162177315 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b80fca50 0x7f25b80fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 --2- 172.21.15.107:0/3162177315 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b81028e0 0x7f25b8102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 -- 172.21.15.107:0/3162177315 >> 172.21.15.107:0/3162177315 conn(0x7f25b8073660 msgr2=0x7f25b8073a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 -- 172.21.15.107:0/3162177315 shutdown_connections 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.212+0000 7f25bddfa640 1 -- 172.21.15.107:0/3162177315 wait complete. 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 Processor -- start 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 -- start start 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8112ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b81028e0 0x7f25b810cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b810d100 con 0x7f25b80fd3e0 2024-09-10T18:53:14.219 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25b810d270 con 0x7f25b81028e0 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25bddfa640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25b810d430 con 0x7f25b80fca50 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8112ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:55272/0 (socket says 172.21.15.107:55272) 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b77fe640 1 -- 172.21.15.107:0/1870750780 learned_addr learned my addr 172.21.15.107:0/1870750780 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b7fff640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b81028e0 0x7f25b810cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 -- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 msgr2=0x7f25b8112960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 -- 172.21.15.107:0/1870750780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b81028e0 msgr2=0x7f25b810cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b81028e0 0x7f25b810cb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b6ffd640 1 -- 172.21.15.107:0/1870750780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25a8005040 con 0x7f25b80fd3e0 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.213+0000 7f25b77fe640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:14.220 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f25b6ffd640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8112ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0028a0 tx=0x7f25ac004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac007c30 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f25ac00f040 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25b00098b0 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25ac009340 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.214+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25b81197f0 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.215+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f25b80fe000 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.215+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7f25ac009770 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.216+0000 7f259ffff640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 0x7f25a00712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.216+0000 7f25b77fe640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 0x7f25a00712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.216+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f25ac08dd00 con 0x7f25b80fd3e0 2024-09-10T18:53:14.221 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.216+0000 7f25b77fe640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 0x7f25a00712d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25a8004f40 tx=0x7f25a800b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:14.222 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.220+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f25ac057190 con 0x7f25b80fd3e0 2024-09-10T18:53:14.377 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.375+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f25b810d5f0 con 0x7f25b80fd3e0 2024-09-10T18:53:14.378 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.376+0000 7f259ffff640 1 -- 172.21.15.107:0/1870750780 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v50) v1 ==== 74+0+13259 (secure 0 0 0) 0x7f25b810d5f0 con 0x7f25b80fd3e0 2024-09-10T18:53:14.378 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:53:14.378 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":50,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","created":"2024-09-10T18:48:17.431796+0000","modified":"2024-09-10T18:53:12.951432+0000","last_up_change":"2024-09-10T18:53:10.945089+0000","last_in_change":"2024-09-10T18:52:53.992364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:51:13.408728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"1106f881-aeec-499d-9e94-6d0e2a28a6d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1943775658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1943775658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6809","nonce":1943775658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1943775658}]},"public_addr":"172.21.15.107:6803/1943775658","cluster_addr":"172.21.15.107:6805/1943775658","heartbeat_back_addr":"172.21.15.107:6809/1943775658","heartbeat_front_addr":"172.21.15.107:6807/1943775658","state":["exists","up"]},{"osd":1,"uuid":"7c834ad4-a6e2-408c-87fa-2953804ec22c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":31,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6811","nonce":462447353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6813","nonce":462447353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6817","nonce":462447353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6815","nonce":462447353}]},"public_addr":"172.21.15.107:6811/462447353","cluster_addr":"172.21.15.107:6813/462447353","heartbeat_back_addr":"172.21.15.107:6817/462447353","heartbeat_front_addr":"172.21.15.107:6815/462447353","state":["exists","up"]},{"osd":2,"uuid":"c61bf972-04c3-4f95-b23f-89662ed8c66d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6819","nonce":1138865377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6821","nonce":1138865377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6825","nonce":1138865377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6823","nonce":1138865377}]},"public_addr":"172.21.15.107:6819/1138865377","cluster_addr":"172.21.15.107:6821/1138865377","heartbeat_back_addr":"172.21.15.107:6825/1138865377","heartbeat_front_addr":"172.21.15.107:6823/1138865377","state":["exists","up"]},{"osd":3,"uuid":"b42bf6e2-ce5b-446e-82e7-ea673401b1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6827","nonce":3842970087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6829","nonce":3842970087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6833","nonce":3842970087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6831","nonce":3842970087}]},"public_addr":"172.21.15.107:6827/3842970087","cluster_addr":"172.21.15.107:6829/3842970087","heartbeat_back_addr":"172.21.15.107:6833/3842970087","heartbeat_front_addr":"172.21.15.107:6831/3842970087","state":["exists","up"]},{"osd":4,"uuid":"6de68f17-c6b5-4b80-8a69-e27860c86651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6801","nonce":335919521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6803","nonce":335919521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6807","nonce":335919521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6805","nonce":335919521}]},"public_addr":"172.21.15.162:6801/335919521","cluster_addr":"172.21.15.162:6803/335919521","heartbeat_back_addr":"172.21.15.162:6807/335919521","heartbeat_front_addr":"172.21.15.162:6805/335919521","state":["exists","up"]},{"osd":5,"uuid":"88d7424d-a81a-4695-8f1e-cda6b4bc228c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6809","nonce":688086025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6811","nonce":688086025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6815","nonce":688086025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6813","nonce":688086025}]},"public_addr":"172.21.15.162:6809/688086025","cluster_addr":"172.21.15.162:6811/688086025","heartbeat_back_addr":"172.21.15.162:6815/688086025","heartbeat_front_addr":"172.21.15.162:6813/688086025","state":["exists","up"]},{"osd":6,"uuid":"ccaefb3e-2fdf-4af2-a846-6ed2dcae1119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":43,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6817","nonce":1609004877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6819","nonce":1609004877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6823","nonce":1609004877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6821","nonce":1609004877}]},"public_addr":"172.21.15.162:6817/1609004877","cluster_addr":"172.21.15.162:6819/1609004877","heartbeat_back_addr":"172.21.15.162:6823/1609004877","heartbeat_front_addr":"172.21.15.162:6821/1609004877","state":["exists","up"]},{"osd":7,"uuid":"9af4b3a1-c67b-4900-bf78-7c93e7efe0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":49,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6825","nonce":3888567684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6827","nonce":3888567684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6831","nonce":3888567684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6829","nonce":3888567684}]},"public_addr":"172.21.15.162:6825/3888567684","cluster_addr":"172.21.15.162:6827/3888567684","heartbeat_back_addr":"172.21.15.162:6831/3888567684","heartbeat_front_addr":"172.21.15.162:6829/3888567684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:23.427116+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:46.698818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:10.767915+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:34.742041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:56.496900+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:20.395868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:44.954153+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:53:09.043479+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1327904374":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6801/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/3160890405":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/46900690":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/72140394":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/2982976703":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/159250052":"2024-09-11T18:48:40.231412+0000","172.21.15.107:6801/1592664852":"2024-09-11T18:48:40.231412+0000","172.21.15.107:6800/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6800/1592664852":"2024-09-11T18:48:40.231412+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.378+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 msgr2=0x7f25a00712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.378+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 0x7f25a00712d0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f25a8004f40 tx=0x7f25a800b040 comp rx=0 tx=0).stop 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.378+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 msgr2=0x7f25b8112ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.378+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8112ea0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f25ac0028a0 tx=0x7f25ac004570 comp rx=0 tx=0).stop 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 shutdown_connections 2024-09-10T18:53:14.380 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f25a006ee20 0x7f25a00712d0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f25b81028e0 0x7f25b810cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f25b80fd3e0 0x7f25b8112ea0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 --2- 172.21.15.107:0/1870750780 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f25b80fca50 0x7f25b8112960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 >> 172.21.15.107:0/1870750780 conn(0x7f25b8073660 msgr2=0x7f25b80fb2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 shutdown_connections 2024-09-10T18:53:14.381 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:14.379+0000 7f25bddfa640 1 -- 172.21.15.107:0/1870750780 wait complete. 2024-09-10T18:53:14.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:13 smithi162 ceph-mon[33228]: osdmap e50: 8 total, 8 up, 8 in 2024-09-10T18:53:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:13 smithi162 ceph-mon[33228]: pgmap v165: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:14 smithi162 ceph-mon[33228]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:53:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1870750780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:53:15.199 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-10T18:51:13.408728+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}] 2024-09-10T18:53:15.199 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-09-10T18:53:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:14 smithi107 ceph-mon[29404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:53:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1870750780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:53:15.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:14 smithi107 ceph-mon[40355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:53:15.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1870750780' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:53:15.440 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:15 smithi107 ceph-mon[29404]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:16.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:15 smithi107 ceph-mon[40355]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:15 smithi162 ceph-mon[33228]: pgmap v166: 1 pgs: 1 remapped+peering; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- 172.21.15.107:0/2060513221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fd8b0 msgr2=0x7fd92c0fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 --2- 172.21.15.107:0/2060513221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c0fdc80 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fd924004890 tx=0x7fd924009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- 172.21.15.107:0/2060513221 shutdown_connections 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 --2- 172.21.15.107:0/2060513221 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c106e60 0x7fd92c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 --2- 172.21.15.107:0/2060513221 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c0fe250 0x7fd92c106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.135 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 --2- 172.21.15.107:0/2060513221 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c0fdc80 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- 172.21.15.107:0/2060513221 >> 172.21.15.107:0/2060513221 conn(0x7fd92c071c00 msgr2=0x7fd92c072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- 172.21.15.107:0/2060513221 shutdown_connections 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- 172.21.15.107:0/2060513221 wait complete. 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 Processor -- start 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.113+0000 7fd932e6d640 1 -- start start 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c112d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 0x7fd92c10cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c10d540 con 0x7fd92c0fe250 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c10d6b0 con 0x7fd92c0fd8b0 2024-09-10T18:53:17.136 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd92c10d870 con 0x7fd92c106e60 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd92bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:47774/0 (socket says 172.21.15.107:47774) 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 0x7fd92c10cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd92bfff640 1 -- 172.21.15.107:0/3032057817 learned_addr learned my addr 172.21.15.107:0/3032057817 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd930c09640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c112d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 -- 172.21.15.107:0/3032057817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c0fd8b0 msgr2=0x7fd92c112d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c112d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 -- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 msgr2=0x7fd92c1132a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 -- 172.21.15.107:0/3032057817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914005040 con 0x7fd92c106e60 2024-09-10T18:53:17.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd92bfff640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd93140a640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 0x7fd92c10cef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009fd0 tx=0x7fd91c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c00c950 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd924005040 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fd91c014410 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd91c0133c0 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.114+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd92c119bb0 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.115+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd92c065b20 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.116+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fd91c00cab0 con 0x7fd92c106e60 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.116+0000 7fd9297fa640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 0x7fd9180712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.117+0000 7fd930c09640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 0x7fd9180712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:17.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.117+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5283+0+0 (secure 0 0 0) 0x7fd91c08e110 con 0x7fd92c106e60 2024-09-10T18:53:17.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.117+0000 7fd930c09640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 0x7fd9180712d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9240040d0 tx=0x7fd924009da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:17.139 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.121+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd91c04d2a0 con 0x7fd92c106e60 2024-09-10T18:53:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.281+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd92c10bc10 con 0x7fd92c106e60 2024-09-10T18:53:17.284 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.282+0000 7fd9297fa640 1 -- 172.21.15.107:0/3032057817 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v50) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd91c057520 con 0x7fd92c106e60 2024-09-10T18:53:17.284 INFO:teuthology.orchestra.run.smithi107.stdout:pg_num: 1 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.283+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 msgr2=0x7fd9180712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.283+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 0x7fd9180712d0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fd9240040d0 tx=0x7fd924009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.283+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 msgr2=0x7fd92c10cef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.283+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 0x7fd92c10cef0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fd91c009fd0 tx=0x7fd91c009780 comp rx=0 tx=0).stop 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 shutdown_connections 2024-09-10T18:53:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fd91806ee20 0x7fd9180712d0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd92c106e60 0x7fd92c10cef0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd92c0fe250 0x7fd92c1132a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 --2- 172.21.15.107:0/3032057817 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd92c0fd8b0 0x7fd92c112d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 >> 172.21.15.107:0/3032057817 conn(0x7fd92c071c00 msgr2=0x7fd92c0fa0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 shutdown_connections 2024-09-10T18:53:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:17.284+0000 7fd932e6d640 1 -- 172.21.15.107:0/3032057817 wait complete. 2024-09-10T18:53:17.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:17 smithi107 ceph-mon[29404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:53:17.588 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:17 smithi107 ceph-mon[29404]: Cluster is now healthy 2024-09-10T18:53:17.588 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:17 smithi107 ceph-mon[40355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:53:17.589 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:17 smithi107 ceph-mon[40355]: Cluster is now healthy 2024-09-10T18:53:17.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:17 smithi162 ceph-mon[33228]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:53:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:17 smithi162 ceph-mon[33228]: Cluster is now healthy 2024-09-10T18:53:17.913 INFO:tasks.cephadm:Adding ceph.rgw.foo.a on smithi107 2024-09-10T18:53:17.914 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply rgw foo.a --placement '1;smithi107=foo.a' 2024-09-10T18:53:18.171 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:18.550 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:18 smithi162 ceph-mon[33228]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:18.550 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3032057817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:53:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:18 smithi107 ceph-mon[29404]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3032057817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:53:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:18 smithi107 ceph-mon[40355]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3032057817' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 -- 172.21.15.162:0/3174830879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 msgr2=0x7fcd1c103c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3174830879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c103c80 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fcd14009cd0 tx=0x7fcd14004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 -- 172.21.15.162:0/3174830879 shutdown_connections 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3174830879 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c103c80 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3174830879 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c0fca30 0x7fcd1c0fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3174830879 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fb760 0x7fcd1c0fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:19.923 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 -- 172.21.15.162:0/3174830879 >> 172.21.15.162:0/3174830879 conn(0x7fcd1c0f6f20 msgr2=0x7fcd1c0f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 -- 172.21.15.162:0/3174830879 shutdown_connections 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 -- 172.21.15.162:0/3174830879 wait complete. 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.903+0000 7fcd213bb640 1 Processor -- start 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 -- start start 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c0fb760 0x7fcd1c10cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fca30 0x7fcd1c10d0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1146b0 con 0x7fcd1c0fca30 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c114820 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd213bb640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd1c1149e0 con 0x7fcd1c0fb760 2024-09-10T18:53:19.924 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:32976/0 (socket says 172.21.15.162:32976) 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 -- 172.21.15.162:0/3667695104 learned_addr learned my addr 172.21.15.162:0/3667695104 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd127fc640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fca30 0x7fcd1c10d0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 -- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c0fb760 msgr2=0x7fcd1c10cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c0fb760 0x7fcd1c10cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 -- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fca30 msgr2=0x7fcd1c10d0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fca30 0x7fcd1c10d0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 -- 172.21.15.162:0/3667695104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd140098b0 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.904+0000 7fcd1b7fe640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402a800 tx=0x7fcd14004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.905+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd14038070 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.925 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.905+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fcd14009250 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.905+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd1c114c60 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.905+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1403cbd0 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.905+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1c115130 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.906+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd1c04bf60 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.906+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fcd14030070 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.907+0000 7fcd13fff640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 0x7fcd08071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.907+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5283+0+0 (secure 0 0 0) 0x7fcd140aecc0 con 0x7fcd1c0fd3e0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.907+0000 7fcd1affd640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 0x7fcd08071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.907+0000 7fcd1affd640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 0x7fcd08071300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097e0 tx=0x7fcd0c006d20 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:19.926 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:19.911+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fcd14078020 con 0x7fcd1c0fd3e0 2024-09-10T18:53:20.078 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.077+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fcd1c106530 con 0x7fcd0806ee50 2024-09-10T18:53:20.085 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.084+0000 7fcd13fff640 1 -- 172.21.15.162:0/3667695104 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+30 (secure 0 0 0) 0x7fcd1c106530 con 0x7fcd0806ee50 2024-09-10T18:53:20.085 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled rgw.foo.a update... 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 msgr2=0x7fcd08071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 0x7fcd08071300 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097e0 tx=0x7fcd0c006d20 comp rx=0 tx=0).stop 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 msgr2=0x7fcd1c1140e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fcd1402a800 tx=0x7fcd14004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 shutdown_connections 2024-09-10T18:53:20.087 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fcd0806ee50 0x7fcd08071300 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fcd1c0fd3e0 0x7fcd1c1140e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fcd1c0fca30 0x7fcd1c10d0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 --2- 172.21.15.162:0/3667695104 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fcd1c0fb760 0x7fcd1c10cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 >> 172.21.15.162:0/3667695104 conn(0x7fcd1c0f6f20 msgr2=0x7fcd1c0fa270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 shutdown_connections 2024-09-10T18:53:20.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:20.086+0000 7fcd213bb640 1 -- 172.21.15.162:0/3667695104 wait complete. 2024-09-10T18:53:20.393 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.405 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:20 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:20.516 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.517 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]: dispatch 2024-09-10T18:53:20.518 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.rgw.foo.a", "caps": ["mon", "allow *", "mgr", "allow rw", "osd", "allow rwx tag rgw *=*"]}]': finished 2024-09-10T18:53:20.518 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:20.518 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:20 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:20.752 DEBUG:teuthology.orchestra.run.smithi107:rgw.foo.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@rgw.foo.a.service 2024-09-10T18:53:20.755 INFO:tasks.cephadm:Adding ceph.iscsi.iscsi.a on smithi162 2024-09-10T18:53:20.755 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd pool create datapool 3 3 replicated 2024-09-10T18:53:21.001 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:21.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:21 smithi162 ceph-mon[33228]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:21 smithi162 ceph-mon[33228]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:21.362 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:21 smithi162 ceph-mon[33228]: Deploying daemon rgw.foo.a on smithi107 2024-09-10T18:53:21.713 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[29404]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.714 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[29404]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:21.714 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[29404]: Deploying daemon rgw.foo.a on smithi107 2024-09-10T18:53:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[40355]: from='client.24292 -' entity='client.admin' cmd=[{"prefix": "orch apply rgw", "svc_id": "foo.a", "placement": "1;smithi107=foo.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[40355]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:21 smithi107 ceph-mon[40355]: Deploying daemon rgw.foo.a on smithi107 2024-09-10T18:53:22.392 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:22 smithi107 ceph-mon[29404]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:53:22.392 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:22 smithi107 ceph-mon[40355]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:53:22.392 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 18:53:22 smithi107 podman[69358]: 2024-09-10 18:53:22.202046042 +0000 UTC m=+0.015425679 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:53:22.479 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:22 smithi162 ceph-mon[33228]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 63 KiB/s, 0 objects/s recovering 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.693+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/945804114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f00ff990 msgr2=0x7fc8f0101d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.693+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/945804114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f00ff990 0x7fc8f0101d70 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fc8e0004950 tx=0x7fc8e0009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.693+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/945804114 shutdown_connections 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/945804114 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f01024a0 0x7fc8f0104950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/945804114 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f00ff990 0x7fc8f0101d70 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/945804114 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f00ff450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/945804114 >> 172.21.15.162:0/945804114 conn(0x7fc8f00f6f20 msgr2=0x7fc8f00f9330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:22.711 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/945804114 shutdown_connections 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/945804114 wait complete. 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 Processor -- start 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- start start 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f010cb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f00ff990 0x7fc8f010d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f01146c0 con 0x7fc8f0066c80 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f0114830 con 0x7fc8f01024a0 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f7cb5640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8f01149f0 con 0x7fc8f00ff990 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f6252640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f5a51640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f010cb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:22.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f6252640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:32986/0 (socket says 172.21.15.162:32986) 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f6252640 1 -- 172.21.15.162:0/1104518117 learned_addr learned my addr 172.21.15.162:0/1104518117 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f5a51640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f010cb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.162:38254/0 (socket says 172.21.15.162:38254) 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.694+0000 7fc8f5250640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f00ff990 0x7fc8f010d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 -- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f00ff990 msgr2=0x7fc8f010d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f00ff990 0x7fc8f010d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 -- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 msgr2=0x7fc8f010cb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f010cb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 -- 172.21.15.162:0/1104518117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8e0005040 con 0x7fc8f01024a0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f6252640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800ece0 tx=0x7fc8e800b040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e800ee40 con 0x7fc8f01024a0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc8e800f040 con 0x7fc8f01024a0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8f0114cd0 con 0x7fc8f01024a0 2024-09-10T18:53:22.713 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8009340 con 0x7fc8f01024a0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.695+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8f0115170 con 0x7fc8f01024a0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.696+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8f004bf60 con 0x7fc8f01024a0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.697+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fc8e8007890 con 0x7fc8f01024a0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.697+0000 7fc8e67fc640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 0x7fc8dc071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.697+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 5283+0+0 (secure 0 0 0) 0x7fc8e808e590 con 0x7fc8f01024a0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.698+0000 7fc8f5a51640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 0x7fc8dc071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.699+0000 7fc8f5a51640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 0x7fc8dc071300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00669e0 tx=0x7fc8ec00b040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:22.714 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.701+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc8e80579a0 con 0x7fc8f01024a0 2024-09-10T18:53:22.865 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:22.864+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"} v 0) v1 -- 0x7fc8f01075c0 con 0x7fc8f01024a0 2024-09-10T18:53:22.965 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 18:53:22 smithi107 podman[69358]: 2024-09-10 18:53:22.492182695 +0000 UTC m=+0.305562314 container init ee9714b2af9e7acce0df5273795e7f80261df78691a52e24db5e167253acd8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2) 2024-09-10T18:53:22.965 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 18:53:22 smithi107 podman[69358]: 2024-09-10 18:53:22.496690544 +0000 UTC m=+0.310070171 container start ee9714b2af9e7acce0df5273795e7f80261df78691a52e24db5e167253acd8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-10T18:53:22.965 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 18:53:22 smithi107 bash[69358]: ee9714b2af9e7acce0df5273795e7f80261df78691a52e24db5e167253acd8f7 2024-09-10T18:53:22.966 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 18:53:22 smithi107 systemd[1]: Started Ceph rgw.foo.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:53:23.276 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.275+0000 7fc8e67fc640 1 -- 172.21.15.162:0/1104518117 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]=0 pool 'datapool' created v51) v1 ==== 160+0+0 (secure 0 0 0) 0x7fc8f01075c0 con 0x7fc8f01024a0 2024-09-10T18:53:23.277 INFO:teuthology.orchestra.run.smithi162.stderr:pool 'datapool' created 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 msgr2=0x7fc8dc071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 0x7fc8dc071300 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc8f00669e0 tx=0x7fc8ec00b040 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 msgr2=0x7fc8f01140f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fc8e800ece0 tx=0x7fc8e800b040 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 shutdown_connections 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc8dc06ee50 0x7fc8dc071300 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc8f01024a0 0x7fc8f01140f0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc8f00ff990 0x7fc8f010d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 --2- 172.21.15.162:0/1104518117 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc8f0066c80 0x7fc8f010cb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:23.281 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 >> 172.21.15.162:0/1104518117 conn(0x7fc8f00f6f20 msgr2=0x7fc8f00fd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:23.282 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 shutdown_connections 2024-09-10T18:53:23.282 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:23.280+0000 7fc8f7cb5640 1 -- 172.21.15.162:0/1104518117 wait complete. 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/1104518117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:53:23.915 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:23.916 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:23.942 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- rbd pool init datapool 2024-09-10T18:53:24.186 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:24.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/1104518117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:53:24.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:53:24.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:24.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:24.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: Saving service rgw.foo.a spec with placement smithi107=foo.a;count:1 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:24.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/1104518117' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:24.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]: dispatch 2024-09-10T18:53:24.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "datapool", "pg_num": 3, "pgp_num": 3, "pool_type": "replicated"}]': finished 2024-09-10T18:53:24.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: osdmap e51: 8 total, 8 up, 8 in 2024-09-10T18:53:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:24.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]: dispatch 2024-09-10T18:53:24.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:53:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:24 smithi162 ceph-mon[33228]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:53:25.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T18:53:25.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:25.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[29404]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 56 KiB/s, 0 objects/s recovering 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:25.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": ".rgw.root","app": "rgw"}]': finished 2024-09-10T18:53:25.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:24 smithi107 ceph-mon[40355]: osdmap e52: 8 total, 8 up, 8 in 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: Checking dashboard <-> RGW credentials 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: pgmap v173: 36 pgs: 5 creating+peering, 23 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:25 smithi162 ceph-mon[33228]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: Checking dashboard <-> RGW credentials 2024-09-10T18:53:26.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: pgmap v173: 36 pgs: 5 creating+peering, 23 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[29404]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: Checking dashboard <-> RGW credentials 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: pgmap v173: 36 pgs: 5 creating+peering, 23 unknown, 8 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: osdmap e53: 8 total, 8 up, 8 in 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:26.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:25 smithi107 ceph-mon[40355]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]: dispatch 2024-09-10T18:53:27.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/3214229721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:53:27.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:26 smithi162 ceph-mon[33228]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/3214229721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[29404]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/3214229721' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]: dispatch 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.log","app": "rgw"}]': finished 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "datapool","app": "rbd"}]': finished 2024-09-10T18:53:27.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:26 smithi107 ceph-mon[40355]: osdmap e54: 8 total, 8 up, 8 in 2024-09-10T18:53:28.629 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: pgmap v176: 68 pgs: 5 creating+peering, 30 unknown, 33 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:28.629 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:53:28.629 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.629 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.630 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.630 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.630 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:53:28.630 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:28 smithi162 ceph-mon[33228]: Cluster is now healthy 2024-09-10T18:53:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: pgmap v176: 68 pgs: 5 creating+peering, 30 unknown, 33 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:53:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[29404]: Cluster is now healthy 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: pgmap v176: 68 pgs: 5 creating+peering, 30 unknown, 33 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: osdmap e55: 8 total, 8 up, 8 in 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3917749713' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3827281169' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: from='client.? ' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]: dispatch 2024-09-10T18:53:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-09-10T18:53:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:28 smithi107 ceph-mon[40355]: Cluster is now healthy 2024-09-10T18:53:28.943 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply iscsi datapool admin admin --trusted_ip_list 172.21.15.162 --placement '1;smithi162=iscsi.a' 2024-09-10T18:53:29.188 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:29.557 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.557 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:29 smithi162 ceph-mon[33228]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.557 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:29 smithi162 ceph-mon[33228]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:53:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[29404]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[29404]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:53:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[40355]: from='client.? ' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.control","app": "rgw"}]': finished 2024-09-10T18:53:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:29 smithi107 ceph-mon[40355]: osdmap e56: 8 total, 8 up, 8 in 2024-09-10T18:53:30.606 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:30 smithi162 ceph-mon[33228]: pgmap v179: 100 pgs: 4 creating+activating, 8 creating+peering, 20 unknown, 68 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:53:30.606 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:30 smithi162 ceph-mon[33228]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:53:30.606 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.606 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.607 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[29404]: pgmap v179: 100 pgs: 4 creating+activating, 8 creating+peering, 20 unknown, 68 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:53:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[29404]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:53:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[40355]: pgmap v179: 100 pgs: 4 creating+activating, 8 creating+peering, 20 unknown, 68 active+clean; 577 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T18:53:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[40355]: osdmap e57: 8 total, 8 up, 8 in 2024-09-10T18:53:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]: dispatch 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 -- 172.21.15.162:0/3025280344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fb730 msgr2=0x7fecc80fbb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 --2- 172.21.15.162:0/3025280344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fb730 0x7fecc80fbb40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fecc4004950 tx=0x7fecc4009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 -- 172.21.15.162:0/3025280344 shutdown_connections 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 --2- 172.21.15.162:0/3025280344 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fd3b0 0x7fecc8103c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 --2- 172.21.15.162:0/3025280344 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fca00 0x7fecc80fce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 --2- 172.21.15.162:0/3025280344 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fb730 0x7fecc80fbb40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 -- 172.21.15.162:0/3025280344 >> 172.21.15.162:0/3025280344 conn(0x7fecc80f6f10 msgr2=0x7fecc80f9320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 -- 172.21.15.162:0/3025280344 shutdown_connections 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.832+0000 7fecceb52640 1 -- 172.21.15.162:0/3025280344 wait complete. 2024-09-10T18:53:30.856 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.833+0000 7fecceb52640 1 Processor -- start 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.833+0000 7fecceb52640 1 -- start start 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.833+0000 7fecceb52640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 0x7fecc810ba20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.833+0000 7fecceb52640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 0x7fecc8112fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc8113590 con 0x7fecc80fd3b0 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecc8113700 con 0x7fecc80fca00 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecc81138c0 con 0x7fecc80fb730 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44776/0 (socket says 172.21.15.162:44776) 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 -- 172.21.15.162:0/2895563101 learned_addr learned my addr 172.21.15.162:0/2895563101 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7feccc8ee640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 0x7fecc810ba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7feccd0ef640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 0x7fecc8112fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:30.857 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 -- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 msgr2=0x7fecc810ba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 0x7fecc810ba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 -- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 msgr2=0x7fecc8112fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 0x7fecc8112fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 -- 172.21.15.162:0/2895563101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecc4005040 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7feccc8ee640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 0x7fecc810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7feccd0ef640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 0x7fecc8112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbffff640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c790 tx=0x7fecb8007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb800a640 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fecb800abe0 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecc8113ba0 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecb8012a40 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.834+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecc81140c0 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.835+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecc804bf60 con 0x7fecc80fca00 2024-09-10T18:53:30.858 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.836+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7fecb800a7a0 con 0x7fecc80fca00 2024-09-10T18:53:30.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.836+0000 7fecbd7fa640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 0x7fecb0071300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:30.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.836+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 7128+0+0 (secure 0 0 0) 0x7fecb808e610 con 0x7fecc80fca00 2024-09-10T18:53:30.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.836+0000 7feccc8ee640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 0x7fecb0071300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:30.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.837+0000 7feccc8ee640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 0x7fecb0071300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fecc402d040 tx=0x7fecc4035040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:30.859 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:30.840+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fecb80573a0 con 0x7fecc80fca00 2024-09-10T18:53:31.010 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.008+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.162", "placement": "1;smithi162=iscsi.a", "target": ["mon-mgr", ""]}) v1 -- 0x7fecc805db50 con 0x7fecb006ee50 2024-09-10T18:53:31.018 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.016+0000 7fecbd7fa640 1 -- 172.21.15.162:0/2895563101 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+35 (secure 0 0 0) 0x7fecc805db50 con 0x7fecb006ee50 2024-09-10T18:53:31.018 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled iscsi.datapool update... 2024-09-10T18:53:31.020 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 msgr2=0x7fecb0071300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 0x7fecb0071300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fecc402d040 tx=0x7fecc4035040 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 msgr2=0x7fecc810bf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c790 tx=0x7fecb8007ec0 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 shutdown_connections 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fecb006ee50 0x7fecb0071300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.018+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fecc80fd3b0 0x7fecc8112fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.019+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fecc80fca00 0x7fecc810bf60 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.019+0000 7fecceb52640 1 --2- 172.21.15.162:0/2895563101 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fecc80fb730 0x7fecc810ba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.019+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 >> 172.21.15.162:0/2895563101 conn(0x7fecc80f6f10 msgr2=0x7fecc80fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.019+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 shutdown_connections 2024-09-10T18:53:31.021 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:31.019+0000 7fecceb52640 1 -- 172.21.15.162:0/2895563101 wait complete. 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.669 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.670 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:31 smithi162 ceph-mon[33228]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:53:31.694 INFO:tasks.cephadm:Distributing iscsi-gateway.cfg... 2024-09-10T18:53:31.694 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:53:31.694 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:53:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:53:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[40355]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool application enable","pool": "default.rgw.meta","app": "rgw"}]': finished 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: osdmap e58: 8 total, 8 up, 8 in 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3433829657' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]: dispatch 2024-09-10T18:53:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:31.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.rgw.foo.a' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2320029974' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "default.rgw.meta", "var": "pg_autoscale_bias", "val": "4"}]': finished 2024-09-10T18:53:31.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:31 smithi107 ceph-mon[29404]: osdmap e59: 8 total, 8 up, 8 in 2024-09-10T18:53:31.722 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:53:31.722 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/iscsi-gateway.cfg 2024-09-10T18:53:31.749 DEBUG:teuthology.orchestra.run.smithi162:iscsi.iscsi.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service 2024-09-10T18:53:31.792 INFO:tasks.cephadm:Adding prometheus.a on smithi162 2024-09-10T18:53:31.792 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply prometheus '1;smithi162=a' 2024-09-10T18:53:32.539 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.162", "placement": "1;smithi162=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: Saving service iscsi.datapool spec with placement smithi162=iscsi.a;count:1 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: pgmap v182: 132 pgs: 4 creating+activating, 11 creating+peering, 30 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.759 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:53:32.760 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:53:32.760 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:32.760 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:32 smithi162 ceph-mon[33228]: Deploying daemon iscsi.iscsi.a on smithi162 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.162", "placement": "1;smithi162=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: Saving service iscsi.datapool spec with placement smithi162=iscsi.a;count:1 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: pgmap v182: 132 pgs: 4 creating+activating, 11 creating+peering, 30 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[29404]: Deploying daemon iscsi.iscsi.a on smithi162 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch apply iscsi", "pool": "datapool", "api_user": "admin", "api_password": "admin", "trusted_ip_list": "172.21.15.162", "placement": "1;smithi162=iscsi.a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: Saving service iscsi.datapool spec with placement smithi162=iscsi.a;count:1 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: pgmap v182: 132 pgs: 4 creating+activating, 11 creating+peering, 30 unknown, 87 active+clean; 578 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 6 op/s 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]: dispatch 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "auth get-or-create", "entity": "client.iscsi.iscsi.a", "caps": ["mon", "profile rbd, allow command \"osd blocklist\", allow command \"config-key get\" with \"key\" prefix \"iscsi/\"", "mgr", "allow command \"service status\"", "osd", "allow rwx"]}]': finished 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:32 smithi107 ceph-mon[40355]: Deploying daemon iscsi.iscsi.a on smithi162 2024-09-10T18:53:33.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:33 smithi162 systemd[1]: Starting Ceph iscsi.iscsi.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:53:34.536 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.523+0000 7f8adc2e8640 1 -- 172.21.15.162:0/33248067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fca30 msgr2=0x7f8ad40fcea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.523+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/33248067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fca30 0x7f8ad40fcea0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004950 tx=0x7f8ac4009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- 172.21.15.162:0/33248067 shutdown_connections 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/33248067 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 0x7f8ad4103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/33248067 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fca30 0x7f8ad40fcea0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/33248067 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fb760 0x7f8ad40fbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- 172.21.15.162:0/33248067 >> 172.21.15.162:0/33248067 conn(0x7f8ad40f6f20 msgr2=0x7f8ad40f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- 172.21.15.162:0/33248067 shutdown_connections 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- 172.21.15.162:0/33248067 wait complete. 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 Processor -- start 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- start start 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:34.537 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fca30 0x7f8ad40fdf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 0x7f8ad40fe4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40febe0 con 0x7f8ad40fd3e0 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad40fed50 con 0x7f8ad40fb760 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8adc2e8640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ad4116ca0 con 0x7f8ad40fca30 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8ada084640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8ada084640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:44792/0 (socket says 172.21.15.162:44792) 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8ada084640 1 -- 172.21.15.162:0/3864230489 learned_addr learned my addr 172.21.15.162:0/3864230489 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.524+0000 7f8ada885640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 0x7f8ad40fe4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ad9883640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fca30 0x7f8ad40fdf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 -- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fca30 msgr2=0x7f8ad40fdf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fca30 0x7f8ad40fdf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 -- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 msgr2=0x7f8ad40fe4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:34.538 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 0x7f8ad40fe4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 -- 172.21.15.162:0/3864230489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac4005040 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8ada084640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c790 tx=0x7f8ad0007ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad000a5e0 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8ad000ab80 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ad4116f00 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad0012960 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.525+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ad4117370 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.526+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ad4107d10 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.527+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 15) v1 ==== 90781+0+0 (secure 0 0 0) 0x7f8ad000a740 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.527+0000 7f8acaffd640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 0x7f8ac00712d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.527+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f8ad008e460 con 0x7f8ad40fb760 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.527+0000 7f8ad9883640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 0x7f8ac00712d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.528+0000 7f8ad9883640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 0x7f8ac00712d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004920 tx=0x7f8ac4035040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:34.539 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.531+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f8ad0057290 con 0x7f8ad40fb760 2024-09-10T18:53:34.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:34 smithi162 ceph-mon[33228]: pgmap v184: 132 pgs: 4 creating+activating, 3 creating+peering, 7 unknown, 118 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 4.7 KiB/s wr, 104 op/s 2024-09-10T18:53:34.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:34 smithi162 podman[62779]: 2024-09-10 18:53:34.32157671 +0000 UTC m=+0.248814681 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:53:34.662 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:34 smithi162 podman[62779]: 2024-09-10 18:53:34.661575651 +0000 UTC m=+0.588813615 container create e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T18:53:34.702 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.700+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f8ad4066670 con 0x7f8ac006ee20 2024-09-10T18:53:34.709 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.707+0000 7f8acaffd640 1 -- 172.21.15.162:0/3864230489 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+31 (secure 0 0 0) 0x7f8ad4066670 con 0x7f8ac006ee20 2024-09-10T18:53:34.709 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled prometheus update... 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 msgr2=0x7f8ac00712d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 0x7f8ac00712d0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f8ac4004920 tx=0x7f8ac4035040 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 msgr2=0x7f8ad40ff890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f8ad000c790 tx=0x7f8ad0007ec0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 shutdown_connections 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f8ac006ee20 0x7f8ac00712d0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.709+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8ad40fd3e0 0x7f8ad40fe4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.710+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8ad40fca30 0x7f8ad40fdf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.710+0000 7f8adc2e8640 1 --2- 172.21.15.162:0/3864230489 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8ad40fb760 0x7f8ad40ff890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.710+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 >> 172.21.15.162:0/3864230489 conn(0x7f8ad40f6f20 msgr2=0x7f8ad4065f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.710+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 shutdown_connections 2024-09-10T18:53:34.712 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:34.710+0000 7f8adc2e8640 1 -- 172.21.15.162:0/3864230489 wait complete. 2024-09-10T18:53:34.769 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:34 smithi107 ceph-mon[29404]: pgmap v184: 132 pgs: 4 creating+activating, 3 creating+peering, 7 unknown, 118 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 4.7 KiB/s wr, 104 op/s 2024-09-10T18:53:34.769 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:34 smithi107 ceph-mon[40355]: pgmap v184: 132 pgs: 4 creating+activating, 3 creating+peering, 7 unknown, 118 active+clean; 580 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 44 KiB/s rd, 4.7 KiB/s wr, 104 op/s 2024-09-10T18:53:35.519 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:35 smithi162 podman[62779]: 2024-09-10 18:53:35.249046413 +0000 UTC m=+1.176284385 container init e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T18:53:35.519 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:35 smithi162 podman[62779]: 2024-09-10 18:53:35.254973733 +0000 UTC m=+1.182211698 container start e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T18:53:35.519 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:35 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu[62875]: log file path now is '/var/log/tcmu-runner.log' 2024-09-10T18:53:35.519 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:35 smithi162 bash[62779]: e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d 2024-09-10T18:53:35.869 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:35 smithi162 ceph-mon[33228]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:35.869 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:35 smithi162 ceph-mon[33228]: Saving service prometheus spec with placement smithi162=a;count:1 2024-09-10T18:53:35.869 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:35 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:35.908 DEBUG:teuthology.orchestra.run.smithi162:prometheus.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@prometheus.a.service 2024-09-10T18:53:35.910 INFO:tasks.cephadm:Adding node-exporter.a on smithi107 2024-09-10T18:53:35.911 INFO:tasks.cephadm:Adding node-exporter.b on smithi162 2024-09-10T18:53:35.911 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply node-exporter '2;smithi107=a;smithi162=b' 2024-09-10T18:53:35.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[40355]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:35.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[40355]: Saving service prometheus spec with placement smithi162=a;count:1 2024-09-10T18:53:35.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[29404]: from='client.24355 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "prometheus", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[29404]: Saving service prometheus spec with placement smithi162=a;count:1 2024-09-10T18:53:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:35 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:36.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 podman[62959]: 2024-09-10 18:53:35.953653863 +0000 UTC m=+0.017281235 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T18:53:36.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 podman[62959]: 2024-09-10 18:53:36.086284759 +0000 UTC m=+0.149912122 container create 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T18:53:36.439 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:36.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 podman[62959]: 2024-09-10 18:53:36.297132043 +0000 UTC m=+0.360759413 container init 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-09-10T18:53:36.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 podman[62959]: 2024-09-10 18:53:36.302881853 +0000 UTC m=+0.366509216 container start 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=) 2024-09-10T18:53:36.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 bash[62959]: 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc 2024-09-10T18:53:36.482 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 systemd[1]: Started Ceph iscsi.iscsi.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:53:36.869 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: pgmap v185: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 71 KiB/s rd, 6.3 KiB/s wr, 172 op/s 2024-09-10T18:53:36.870 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:36.870 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:36.870 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:36.870 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:36.870 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:36.870 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Started the configuration object watcher 2024-09-10T18:53:36.870 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Checking for config object changes every 1s 2024-09-10T18:53:36.870 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Processing osd blocklist entries for this node 2024-09-10T18:53:36.999 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: pgmap v185: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 71 KiB/s rd, 6.3 KiB/s wr, 172 op/s 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: pgmap v185: 132 pgs: 132 active+clean; 581 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 71 KiB/s rd, 6.3 KiB/s wr, 172 op/s 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.000 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:37.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:36 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Reading the configuration object to update local LIO configuration 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Configuration does not have an entry for this host(smithi162.front.sepia.ceph.com) - nothing to define to LIO 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: * Serving Flask app 'rbd-target-api' (lazy loading) 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: * Environment: production 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: Use a production WSGI server instead. 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: * Debug mode: off 2024-09-10T18:53:37.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug * Running on all addresses. 2024-09-10T18:53:37.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:53:37.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: * Running on all addresses. 2024-09-10T18:53:37.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: WARNING: This is a development server. Do not use it in a production deployment. 2024-09-10T18:53:37.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug * Running on http://[fe80::ec4:7aff:fe88:7f41]:5000/ (Press CTRL+C to quit) 2024-09-10T18:53:37.175 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: * Running on http://[fe80::ec4:7aff:fe88:7f41]:5000/ (Press CTRL+C to quit) 2024-09-10T18:53:38.117 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:37 smithi162 ceph-mon[33228]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:53:38.117 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/2859408969' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:53:38.118 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:37 smithi162 ceph-mon[33228]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:53:38.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[29404]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:53:38.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/2859408969' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:53:38.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[29404]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:53:38.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[40355]: Checking pool "datapool" exists for service iscsi.datapool 2024-09-10T18:53:38.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/2859408969' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-09-10T18:53:38.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:37 smithi107 ceph-mon[40355]: osdmap e60: 8 total, 8 up, 8 in 2024-09-10T18:53:38.754 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 -- 172.21.15.162:0/1131309199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fb760 msgr2=0x7fe8c80fbb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 --2- 172.21.15.162:0/1131309199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fb760 0x7fe8c80fbb70 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe8b8004950 tx=0x7fe8b8009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 -- 172.21.15.162:0/1131309199 shutdown_connections 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 --2- 172.21.15.162:0/1131309199 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 0x7fe8c8103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 --2- 172.21.15.162:0/1131309199 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fca30 0x7fe8c80fcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 --2- 172.21.15.162:0/1131309199 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fb760 0x7fe8c80fbb70 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 -- 172.21.15.162:0/1131309199 >> 172.21.15.162:0/1131309199 conn(0x7fe8c80f6f20 msgr2=0x7fe8c80f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 -- 172.21.15.162:0/1131309199 shutdown_connections 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 -- 172.21.15.162:0/1131309199 wait complete. 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.396+0000 7fe8ce793640 1 Processor -- start 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 -- start start 2024-09-10T18:53:38.755 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fb760 0x7fe8c80ff6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 0x7fe8c80fe2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80fe9f0 con 0x7fe8c80fd3e0 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c80feb60 con 0x7fe8c80fca30 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ce793640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8c8116ca0 con 0x7fe8c80fb760 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34100/0 (socket says 172.21.15.162:34100) 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 -- 172.21.15.162:0/2173580013 learned_addr learned my addr 172.21.15.162:0/2173580013 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8ccd30640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 0x7fe8c80fe2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8c7fff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fb760 0x7fe8c80ff6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 -- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fb760 msgr2=0x7fe8c80ff6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:38.756 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fb760 0x7fe8c80ff6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 -- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 msgr2=0x7fe8c80fe2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 0x7fe8c80fe2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 -- 172.21.15.162:0/2173580013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8b8005040 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.397+0000 7fe8bffff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c890 tx=0x7fe8b0007f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.398+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b000ce20 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.398+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fe8b000ac30 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.398+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8c8116f00 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.398+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8b0012b80 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.398+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8c8117390 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.399+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8c8106b50 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.399+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 90832+0+0 (secure 0 0 0) 0x7fe8b000a7f0 con 0x7fe8c80fca30 2024-09-10T18:53:38.757 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.400+0000 7fe8c57fa640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 0x7fe8b4071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.400+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7fe8b008e580 con 0x7fe8c80fca30 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled node-exporter update... 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.400+0000 7fe8c7fff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 0x7fe8b4071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.400+0000 7fe8c7fff640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 0x7fe8b4071360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8b802d040 tx=0x7fe8b8035040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.404+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe8b0057360 con 0x7fe8c80fca30 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.575+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi162=b", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8c8065a10 con 0x7fe8b406eeb0 2024-09-10T18:53:38.758 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.584+0000 7fe8c57fa640 1 -- 172.21.15.162:0/2173580013 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+34 (secure 0 0 0) 0x7fe8c8065a10 con 0x7fe8b406eeb0 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 msgr2=0x7fe8b4071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 0x7fe8b4071360 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fe8b802d040 tx=0x7fe8b8035040 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 msgr2=0x7fe8c80fdd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fe8b000c890 tx=0x7fe8b0007f50 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 shutdown_connections 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fe8b406eeb0 0x7fe8b4071360 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8c80fd3e0 0x7fe8c80fe2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8c80fca30 0x7fe8c80fdd70 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 --2- 172.21.15.162:0/2173580013 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8c80fb760 0x7fe8c80ff6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 >> 172.21.15.162:0/2173580013 conn(0x7fe8c80f6f20 msgr2=0x7fe8c8065300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:38.759 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 shutdown_connections 2024-09-10T18:53:38.760 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:38.586+0000 7fe8ce793640 1 -- 172.21.15.162:0/2173580013 wait complete. 2024-09-10T18:53:39.015 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:38 smithi162 ceph-mon[33228]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 163 op/s 2024-09-10T18:53:39.015 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:38 smithi162 ceph-mon[33228]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:53:39.015 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:38 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.016 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:38 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[29404]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 163 op/s 2024-09-10T18:53:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[29404]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:53:39.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[40355]: pgmap v187: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 67 KiB/s rd, 5.4 KiB/s wr, 163 op/s 2024-09-10T18:53:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[40355]: mgrmap e16: y(active, since 4m), standbys: x 2024-09-10T18:53:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:38 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:39.308 DEBUG:teuthology.orchestra.run.smithi107:node-exporter.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.a.service 2024-09-10T18:53:39.311 DEBUG:teuthology.orchestra.run.smithi162:node-exporter.b> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.b.service 2024-09-10T18:53:39.313 INFO:tasks.cephadm:Adding alertmanager.a on smithi107 2024-09-10T18:53:39.313 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply alertmanager '1;smithi107=a' 2024-09-10T18:53:40.025 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:39 smithi162 ceph-mon[33228]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi162=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:40.025 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:39 smithi162 ceph-mon[33228]: Saving service node-exporter spec with placement smithi107=a;smithi162=b;count:2 2024-09-10T18:53:40.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:39 smithi107 ceph-mon[29404]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi162=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:40.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:39 smithi107 ceph-mon[29404]: Saving service node-exporter spec with placement smithi107=a;smithi162=b;count:2 2024-09-10T18:53:40.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:39 smithi107 ceph-mon[40355]: from='client.24391 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "node-exporter", "placement": "2;smithi107=a;smithi162=b", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:40.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:39 smithi107 ceph-mon[40355]: Saving service node-exporter spec with placement smithi107=a;smithi162=b;count:2 2024-09-10T18:53:40.317 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:41.159 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.6 KiB/s wr, 140 op/s 2024-09-10T18:53:41.159 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.160 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.160 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:41.160 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:41.160 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:40 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.6 KiB/s wr, 140 op/s 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: pgmap v188: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 58 KiB/s rd, 4.6 KiB/s wr, 140 op/s 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:53:41.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:53:41.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:40 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- 172.21.15.162:0/3787242704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 msgr2=0x7f23600fcc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 --2- 172.21.15.162:0/3787242704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f23600fcc70 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f2350009cd0 tx=0x7f2350004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- 172.21.15.162:0/3787242704 shutdown_connections 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 --2- 172.21.15.162:0/3787242704 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360104d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 --2- 172.21.15.162:0/3787242704 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f23600fdfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 --2- 172.21.15.162:0/3787242704 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f23600fcc70 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:41.913 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- 172.21.15.162:0/3787242704 >> 172.21.15.162:0/3787242704 conn(0x7f23600f8040 msgr2=0x7f23600fa450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- 172.21.15.162:0/3787242704 shutdown_connections 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- 172.21.15.162:0/3787242704 wait complete. 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 Processor -- start 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.891+0000 7f2366b0d640 1 -- start start 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f236010d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360114140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360114710 con 0x7f23600fdb30 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2360114880 con 0x7f23600fc860 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2360114a40 con 0x7f23600fe4e0 2024-09-10T18:53:41.914 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34120/0 (socket says 172.21.15.162:34120) 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 -- 172.21.15.162:0/2738149693 learned_addr learned my addr 172.21.15.162:0/2738149693 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f235ffff640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f236010d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23650aa640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360114140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 -- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 msgr2=0x7f2360114140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360114140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 -- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 msgr2=0x7f236010d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f236010d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 -- 172.21.15.162:0/2738149693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23500098b0 con 0x7f23600fc860 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23650aa640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f235ffff640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f236010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f23648a9640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2350009d30 tx=0x7f2350004d10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:41.915 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2350038070 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f2350033ce0 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2360114cc0 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f235003c9c0 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.892+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2360115180 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.893+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f236004bf60 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.894+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 90832+0+0 (secure 0 0 0) 0x7f2350030070 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.894+0000 7f235d7fa640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 0x7f234c071390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.894+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f23500aef70 con 0x7f23600fc860 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.894+0000 7f235ffff640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 0x7f234c071390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.895+0000 7f235ffff640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 0x7f234c071390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f2348009e50 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:41.916 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:41.898+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2350077ce0 con 0x7f23600fc860 2024-09-10T18:53:42.076 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.074+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f23601075a0 con 0x7f234c06eee0 2024-09-10T18:53:42.077 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:41 smithi162 ceph-mon[33228]: Deploying daemon prometheus.a on smithi162 2024-09-10T18:53:42.086 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.084+0000 7f235d7fa640 1 -- 172.21.15.162:0/2738149693 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+33 (secure 0 0 0) 0x7f23601075a0 con 0x7f234c06eee0 2024-09-10T18:53:42.086 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled alertmanager update... 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 msgr2=0x7f234c071390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 0x7f234c071390 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f2348009fd0 tx=0x7f2348009e50 comp rx=0 tx=0).stop 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 msgr2=0x7f236010cba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2350009d30 tx=0x7f2350004d10 comp rx=0 tx=0).stop 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 shutdown_connections 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f234c06eee0 0x7f234c071390 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:42.088 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f23600fe4e0 0x7f2360114140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:42.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f23600fdb30 0x7f236010d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:42.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 --2- 172.21.15.162:0/2738149693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f23600fc860 0x7f236010cba0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:42.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 >> 172.21.15.162:0/2738149693 conn(0x7f23600f8040 msgr2=0x7f23600f9950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:42.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 shutdown_connections 2024-09-10T18:53:42.089 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:42.086+0000 7f2366b0d640 1 -- 172.21.15.162:0/2738149693 wait complete. 2024-09-10T18:53:42.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:41 smithi107 ceph-mon[29404]: Deploying daemon prometheus.a on smithi162 2024-09-10T18:53:42.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:41 smithi107 ceph-mon[40355]: Deploying daemon prometheus.a on smithi162 2024-09-10T18:53:42.817 DEBUG:teuthology.orchestra.run.smithi107:alertmanager.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@alertmanager.a.service 2024-09-10T18:53:42.820 INFO:tasks.cephadm:Adding grafana.a on smithi162 2024-09-10T18:53:42.820 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph orch apply grafana '1;smithi162=a' 2024-09-10T18:53:43.064 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:43.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:42 smithi162 ceph-mon[33228]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.9 KiB/s wr, 115 op/s 2024-09-10T18:53:43.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:42 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:43.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:42 smithi107 ceph-mon[29404]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.9 KiB/s wr, 115 op/s 2024-09-10T18:53:43.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:42 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:43.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:42 smithi107 ceph-mon[40355]: pgmap v189: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 47 KiB/s rd, 3.9 KiB/s wr, 115 op/s 2024-09-10T18:53:43.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:42 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:43 smithi162 ceph-mon[33228]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:43 smithi162 ceph-mon[33228]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-10T18:53:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:43 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:44.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[29404]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:44.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[29404]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-10T18:53:44.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:44.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[40355]: from='client.24397 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "alertmanager", "placement": "1;smithi107=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:44.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[40355]: Saving service alertmanager spec with placement smithi107=a;count:1 2024-09-10T18:53:44.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:43 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:45.110 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:44 smithi162 ceph-mon[33228]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 1.9 KiB/s wr, 64 op/s 2024-09-10T18:53:45.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:44 smithi107 ceph-mon[29404]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 1.9 KiB/s wr, 64 op/s 2024-09-10T18:53:45.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:44 smithi107 ceph-mon[40355]: pgmap v190: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 27 KiB/s rd, 1.9 KiB/s wr, 64 op/s 2024-09-10T18:53:45.471 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 -- 172.21.15.162:0/605976036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fb750 msgr2=0x7f63980fbb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:45.471 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 --2- 172.21.15.162:0/605976036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fb750 0x7f63980fbb60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6388004950 tx=0x7f6388009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.471 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 -- 172.21.15.162:0/605976036 shutdown_connections 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 --2- 172.21.15.162:0/605976036 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fd3d0 0x7f6398103c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 --2- 172.21.15.162:0/605976036 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fca20 0x7f63980fce90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 --2- 172.21.15.162:0/605976036 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fb750 0x7f63980fbb60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 -- 172.21.15.162:0/605976036 >> 172.21.15.162:0/605976036 conn(0x7f63980f6f10 msgr2=0x7f63980f9340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 -- 172.21.15.162:0/605976036 shutdown_connections 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 -- 172.21.15.162:0/605976036 wait complete. 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.401+0000 7f639d61e640 1 Processor -- start 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 -- start start 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fb750 0x7f63980ff690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:45.472 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fca20 0x7f63980fdd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980fe9e0 con 0x7f63980fca20 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63980feb50 con 0x7f63980fd3d0 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f639d61e640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6398116d90 con 0x7f63980fb750 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:34128/0 (socket says 172.21.15.162:34128) 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 -- 172.21.15.162:0/2413344608 learned_addr learned my addr 172.21.15.162:0/2413344608 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f6396ffd640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fb750 0x7f63980ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63967fc640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fca20 0x7f63980fdd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 -- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fb750 msgr2=0x7f63980ff690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fb750 0x7f63980ff690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 -- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fca20 msgr2=0x7f63980fdd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:45.473 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fca20 0x7f63980fdd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 -- 172.21.15.162:0/2413344608 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6388005040 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f63977fe640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f639000db80 tx=0x7f6390004e60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.402+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6390017070 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.403+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f639000f040 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.403+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6398116ff0 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.403+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63900036a0 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.403+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63981173e0 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.404+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f639804bf60 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.404+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 90832+0+0 (secure 0 0 0) 0x7f6390003ae0 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.404+0000 7f638f7fe640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 0x7f6384071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.404+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f639008e800 con 0x7f63980fd3d0 2024-09-10T18:53:45.474 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.405+0000 7f6396ffd640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 0x7f6384071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:45.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.405+0000 7f6396ffd640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 0x7f6384071360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f638802a570 tx=0x7f6388035040 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:45.475 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.408+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f63900573d0 con 0x7f63980fd3d0 2024-09-10T18:53:45.578 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.576+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 --> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}) v1 -- 0x7f6398106540 con 0x7f638406eeb0 2024-09-10T18:53:45.585 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.583+0000 7f638f7fe640 1 -- 172.21.15.162:0/2413344608 <== mgr.14148 v2:172.21.15.107:6800/383804209 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+28 (secure 0 0 0) 0x7f6398106540 con 0x7f638406eeb0 2024-09-10T18:53:45.585 INFO:teuthology.orchestra.run.smithi162.stdout:Scheduled grafana update... 2024-09-10T18:53:45.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 msgr2=0x7f6384071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:45.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 0x7f6384071360 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f638802a570 tx=0x7f6388035040 comp rx=0 tx=0).stop 2024-09-10T18:53:45.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 msgr2=0x7f63980fe2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:45.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f639000db80 tx=0x7f6390004e60 comp rx=0 tx=0).stop 2024-09-10T18:53:45.587 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 shutdown_connections 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f638406eeb0 0x7f6384071360 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f63980fd3d0 0x7f63980fe2a0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f63980fca20 0x7f63980fdd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 --2- 172.21.15.162:0/2413344608 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f63980fb750 0x7f63980ff690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 >> 172.21.15.162:0/2413344608 conn(0x7f63980f6f10 msgr2=0x7f6398066820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 shutdown_connections 2024-09-10T18:53:45.588 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:45.585+0000 7f639d61e640 1 -- 172.21.15.162:0/2413344608 wait complete. 2024-09-10T18:53:46.525 DEBUG:teuthology.orchestra.run.smithi162:grafana.a> sudo journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@grafana.a.service 2024-09-10T18:53:46.528 INFO:tasks.cephadm:Setting up client nodes... 2024-09-10T18:53:46.528 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:53:46.775 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[29404]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[29404]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[29404]: Saving service grafana spec with placement smithi162=a;count:1 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[40355]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[40355]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:46.896 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[40355]: Saving service grafana spec with placement smithi162=a;count:1 2024-09-10T18:53:46.897 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:46 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:46 smithi162 ceph-mon[33228]: pgmap v191: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 4.8 KiB/s rd, 204 B/s wr, 11 op/s 2024-09-10T18:53:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:46 smithi162 ceph-mon[33228]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "grafana", "placement": "1;smithi162=a", "target": ["mon-mgr", ""]}]: dispatch 2024-09-10T18:53:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:46 smithi162 ceph-mon[33228]: Saving service grafana spec with placement smithi162=a;count:1 2024-09-10T18:53:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:46 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:46.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:53:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:53:47.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:47 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:47 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:47 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 -- 172.21.15.107:0/2056189881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fb710 msgr2=0x7f49c40fbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 --2- 172.21.15.107:0/2056189881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fb710 0x7f49c40fbb20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f49b4004890 tx=0x7f49b4009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 -- 172.21.15.107:0/2056189881 shutdown_connections 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 --2- 172.21.15.107:0/2056189881 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fd390 0x7f49c4103c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 --2- 172.21.15.107:0/2056189881 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fc9e0 0x7f49c40fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.627 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 --2- 172.21.15.107:0/2056189881 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fb710 0x7f49c40fbb20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 -- 172.21.15.107:0/2056189881 >> 172.21.15.107:0/2056189881 conn(0x7f49c40f6ed0 msgr2=0x7f49c40f9300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 -- 172.21.15.107:0/2056189881 shutdown_connections 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.617+0000 7f49c980f640 1 -- 172.21.15.107:0/2056189881 wait complete. 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 Processor -- start 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 -- start start 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fb710 0x7f49c40ff600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 0x7f49c40fe260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40fe900 con 0x7f49c40fc9e0 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49c40fea70 con 0x7f49c40fb710 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c980f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49c4116d40 con 0x7f49c40fd390 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c37fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 0x7f49c40fe260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:48.628 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:52512/0 (socket says 172.21.15.107:52512) 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 -- 172.21.15.107:0/462811697 learned_addr learned my addr 172.21.15.107:0/462811697 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49c2ffd640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fb710 0x7f49c40ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 -- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 msgr2=0x7f49c40fe260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 0x7f49c40fe260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.618+0000 7f49ba7fc640 1 -- 172.21.15.107:0/462811697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fb710 msgr2=0x7f49c40ff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49ba7fc640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fb710 0x7f49c40ff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49ba7fc640 1 -- 172.21.15.107:0/462811697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49ac005040 con 0x7f49c40fc9e0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49c37fe640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 0x7f49c40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49ba7fc640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009550 tx=0x7f49ac009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac00e930 con 0x7f49c40fc9e0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f49ac00f040 con 0x7f49c40fc9e0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49b4005040 con 0x7f49c40fc9e0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49ac013b10 con 0x7f49c40fc9e0 2024-09-10T18:53:48.629 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.619+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49c41172c0 con 0x7f49c40fc9e0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.620+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49c404bf60 con 0x7f49c40fc9e0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.620+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 16) v1 ==== 90832+0+0 (secure 0 0 0) 0x7f49ac00ea90 con 0x7f49c40fc9e0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.621+0000 7f49bbfff640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 0x7f49b0071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.621+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f49ac08e5f0 con 0x7f49c40fc9e0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.621+0000 7f49c2ffd640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 0x7f49b0071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.622+0000 7f49c2ffd640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 0x7f49b0071360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009040 tx=0x7f49b4009da0 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:48.630 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.625+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f49ac057240 con 0x7f49c40fc9e0 2024-09-10T18:53:48.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.838+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f49c41174e0 con 0x7f49c40fc9e0 2024-09-10T18:53:48.843 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.842+0000 7f49bbfff640 1 -- 172.21.15.107:0/462811697 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v16) v1 ==== 170+0+59 (secure 0 0 0) 0x7f49ac05aa50 con 0x7f49c40fc9e0 2024-09-10T18:53:48.844 INFO:teuthology.orchestra.run.smithi107.stdout:[client.0] 2024-09-10T18:53:48.844 INFO:teuthology.orchestra.run.smithi107.stdout: key = AQC8leBmTooLMhAALEO7o3lJU1nRvFXKAp+Hvg== 2024-09-10T18:53:48.844 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:48 smithi107 ceph-mon[29404]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 817 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:53:48.844 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:48 smithi107 ceph-mon[40355]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 817 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 msgr2=0x7f49b0071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 0x7f49b0071360 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f49b4009040 tx=0x7f49b4009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 msgr2=0x7f49c40fdd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f49ac009550 tx=0x7f49ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 shutdown_connections 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f49b006eeb0 0x7f49b0071360 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f49c40fd390 0x7f49c40fe260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f49c40fc9e0 0x7f49c40fdd20 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 --2- 172.21.15.107:0/462811697 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f49c40fb710 0x7f49c40ff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.845+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 >> 172.21.15.107:0/462811697 conn(0x7f49c40f6ed0 msgr2=0x7f49c40652c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.846+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 shutdown_connections 2024-09-10T18:53:48.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:48.846+0000 7f49c980f640 1 -- 172.21.15.107:0/462811697 wait complete. 2024-09-10T18:53:48.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:48 smithi162 ceph-mon[33228]: pgmap v192: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 817 B/s rd, 102 B/s wr, 1 op/s 2024-09-10T18:53:49.433 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:53:49.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-10T18:53:49.433 DEBUG:teuthology.orchestra.run.smithi107:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-10T18:53:49.471 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph auth get-or-create client.1 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-10T18:53:49.713 INFO:teuthology.orchestra.run.smithi162.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.b/config 2024-09-10T18:53:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:49.833 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:49.833 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/462811697' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:50.737 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:50 smithi162 ceph-mon[33228]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:50 smithi107 ceph-mon[29404]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:50 smithi107 ceph-mon[40355]: pgmap v193: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 682 B/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:52.876 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:52 smithi162 ceph-mon[33228]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:52 smithi107 ceph-mon[29404]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:52.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:52 smithi107 ceph-mon[40355]: pgmap v194: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 85 B/s wr, 1 op/s 2024-09-10T18:53:54.050 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 -- 172.21.15.162:0/3995100616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 msgr2=0x7f368c0fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 --2- 172.21.15.162:0/3995100616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c0fcc60 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f367c004950 tx=0x7f367c009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 -- 172.21.15.162:0/3995100616 shutdown_connections 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 --2- 172.21.15.162:0/3995100616 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c104d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 --2- 172.21.15.162:0/3995100616 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c0fdf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 --2- 172.21.15.162:0/3995100616 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c0fcc60 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 -- 172.21.15.162:0/3995100616 >> 172.21.15.162:0/3995100616 conn(0x7f368c0f8010 msgr2=0x7f368c0fa440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 -- 172.21.15.162:0/3995100616 shutdown_connections 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.030+0000 7f3691984640 1 -- 172.21.15.162:0/3995100616 wait complete. 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 Processor -- start 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 -- start start 2024-09-10T18:53:54.051 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c10d1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c1141d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c1147a0 con 0x7f368c0fe4d0 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f368c114910 con 0x7f368c0fc850 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f3691984640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f368c114ad0 con 0x7f368c0fdb20 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.162:39060/0 (socket says 172.21.15.162:39060) 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 -- 172.21.15.162:0/234962944 learned_addr learned my addr 172.21.15.162:0/234962944 (peer_addr_for_me v2:172.21.15.162:0/0) 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368a7fc640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c10d1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368b7fe640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c1141d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 -- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 msgr2=0x7f368c10d1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.052 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c10d1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 -- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 msgr2=0x7f368c1141d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c1141d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368affd640 1 -- 172.21.15.162:0/234962944 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f367c005040 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368a7fc640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c10d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.031+0000 7f368b7fe640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f368affd640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f367c02d040 tx=0x7f367c035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c033420 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f367c0339c0 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f368c114d50 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f367c03c6f0 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.032+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f368c1151b0 con 0x7f368c0fc850 2024-09-10T18:53:54.053 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.033+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f368c107cb0 con 0x7f368c0fc850 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.033+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 16) v1 ==== 90832+0+0 (secure 0 0 0) 0x7f367c030020 con 0x7f368c0fc850 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.034+0000 7f36737fe640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 0x7f3674071360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.034+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f367c0ae520 con 0x7f368c0fc850 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.034+0000 7f368a7fc640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 0x7f3674071360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.035+0000 7f368a7fc640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 0x7f3674071360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f3680009e90 comp rx=0 tx=0).ready entity=mgr.14148 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:54.054 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.038+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f367c077250 con 0x7f368c0fc850 2024-09-10T18:53:54.288 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.287+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) v1 -- 0x7f368c103b40 con 0x7f368c0fc850 2024-09-10T18:53:54.293 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.292+0000 7f36737fe640 1 -- 172.21.15.162:0/234962944 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v17) v1 ==== 170+0+59 (secure 0 0 0) 0x7f367c07aa60 con 0x7f368c0fc850 2024-09-10T18:53:54.293 INFO:teuthology.orchestra.run.smithi162.stdout:[client.1] 2024-09-10T18:53:54.293 INFO:teuthology.orchestra.run.smithi162.stdout: key = AQDCleBm3sU8ERAA97qxohJswphSdNWTio9iug== 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 msgr2=0x7f3674071360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 0x7f3674071360 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3680007fd0 tx=0x7f3680009e90 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 msgr2=0x7f368c10cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f367c02d040 tx=0x7f367c035040 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 shutdown_connections 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f367406eeb0 0x7f3674071360 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f368c0fe4d0 0x7f368c1141d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f368c0fdb20 0x7f368c10d1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 --2- 172.21.15.162:0/234962944 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f368c0fc850 0x7f368c10cc70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.294+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 >> 172.21.15.162:0/234962944 conn(0x7f368c0f8010 msgr2=0x7f368c100d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.295+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 shutdown_connections 2024-09-10T18:53:54.296 INFO:teuthology.orchestra.run.smithi162.stderr:2024-09-10T18:53:54.295+0000 7f3691984640 1 -- 172.21.15.162:0/234962944 wait complete. 2024-09-10T18:53:54.888 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:54 smithi162 ceph-mon[33228]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:54.888 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.162:0/234962944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.888 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.888 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:54.912 DEBUG:teuthology.orchestra.run.smithi162:> set -ex 2024-09-10T18:53:54.912 DEBUG:teuthology.orchestra.run.smithi162:> sudo dd of=/etc/ceph/ceph.client.1.keyring 2024-09-10T18:53:54.912 DEBUG:teuthology.orchestra.run.smithi162:> sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-09-10T18:53:54.951 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-10T18:53:54.952 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-10T18:53:54.952 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph mgr dump --format=json 2024-09-10T18:53:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[29404]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.162:0/234962944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[40355]: pgmap v195: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:53:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.162:0/234962944' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-09-10T18:53:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.1", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-10T18:53:55.140 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 podman[64913]: 2024-09-10 18:53:55.121369832 +0000 UTC m=+0.095682066 container create 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:53:55.140 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 podman[64913]: 2024-09-10 18:53:55.04259521 +0000 UTC m=+0.016907454 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:53:55.205 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:55.399 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 podman[64913]: 2024-09-10 18:53:55.348439978 +0000 UTC m=+0.322752214 container init 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:53:55.399 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 podman[64913]: 2024-09-10 18:53:55.353308003 +0000 UTC m=+0.327620238 container start 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:53:55.399 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.377Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.377Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.377Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi162 (none))" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.377Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.377Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.378Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.378Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.380Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.380Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.382Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.382Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.556µs 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.382Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.382Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=0 2024-09-10T18:53:55.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.382Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=34.7µs wal_replay_duration=151.973µs wbl_replay_duration=128ns total_replay_duration=203.595µs 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.383Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.383Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.383Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.398Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.062549ms db_storage=729ns remote_storage=1.563µs web_handler=302ns query_engine=559ns scrape=385.584µs scrape_sd=16.819µs notify=958ns notify_sd=1.375µs rules=14.46207ms tracing=5.72µs 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.398Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:53:55.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:53:55.398Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:53:55.673 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 bash[64913]: 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c 2024-09-10T18:53:55.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:53:55 smithi162 systemd[1]: Started Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[29404]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[40355]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.721 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setuser ceph since I am not root 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ignoring --setgroup ceph since I am not root 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-mgr[29680]: -- 172.21.15.107:0/3636518611 <== mon.2 v2:172.21.15.107:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x56153201a340 con 0x561532027000 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:56.660+0000 7f731a6eeec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:53:56.722 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:56.716+0000 7f731a6eeec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mon[33228]: pgmap v196: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd=[{"prefix": "mgr module enable", "module": "prometheus"}]: dispatch 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ignoring --setuser ceph since I am not root 2024-09-10T18:53:56.783 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ignoring --setgroup ceph since I am not root 2024-09-10T18:53:56.784 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-mgr[35312]: -- 172.21.15.162:0/3989988927 <== mon.2 v2:172.21.15.107:3301/0 4 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 194+0+0 (secure 0 0 0) 0x55ae38098340 con 0x55ae380a4c00 2024-09-10T18:53:56.784 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:56.662+0000 7f598241fec0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-09-10T18:53:56.784 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:56.716+0000 7f598241fec0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-09-10T18:53:56.784 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:56.782+0000 7f598241fec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:53:57.027 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:56.784+0000 7f731a6eeec0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-09-10T18:53:57.027 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:56.912+0000 7f731a6eeec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:53:57.041 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:56.904+0000 7f598241fec0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 -- 172.21.15.107:0/1684471050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 msgr2=0x7f743c101ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1684471050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c101ba0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7430004980 tx=0x7f7430009da0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 -- 172.21.15.107:0/1684471050 shutdown_connections 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1684471050 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fc1b0 0x7f743c108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1684471050 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fb800 0x7f743c0fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1684471050 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c101ba0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 -- 172.21.15.107:0/1684471050 >> 172.21.15.107:0/1684471050 conn(0x7f743c071c00 msgr2=0x7f743c072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:57.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 -- 172.21.15.107:0/1684471050 shutdown_connections 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.025+0000 7f7442d2d640 1 -- 172.21.15.107:0/1684471050 wait complete. 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.026+0000 7f7442d2d640 1 Processor -- start 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.026+0000 7f7442d2d640 1 -- start start 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 0x7f743c1152d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c10f570 con 0x7f743c1017d0 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f743c10f6e0 con 0x7f743c0fb800 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7442d2d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f743c10f8a0 con 0x7f743c0fc1b0 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f74412ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.162:3300/0 says I am v2:172.21.15.107:42554/0 (socket says 172.21.15.107:42554) 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f743bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 0x7f743c1152d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:53:57.063 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 -- 172.21.15.107:0/1948632480 learned_addr learned my addr 172.21.15.107:0/1948632480 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f74412ca640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34898/0 (socket says 172.21.15.107:34898) 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 msgr2=0x7f743c1152d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 0x7f743c1152d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 msgr2=0x7f743c10ef20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 -- 172.21.15.107:0/1948632480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7430005040 con 0x7f743c0fb800 2024-09-10T18:53:57.064 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f743bfff640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 0x7f743c1152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f74412ca640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.027+0000 7f7440ac9640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f743002d040 tx=0x7f7430035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.028+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f743002ab50 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.028+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f743002acb0 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.028+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f743c10fb20 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.028+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7430033e40 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.028+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f743c11bd80 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.029+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f743c0662f0 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.029+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 17) v1 ==== 90846+0+0 (secure 0 0 0) 0x7f7430030020 con 0x7f743c0fb800 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.030+0000 7f74397fa640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 0x7f742c0713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.030+0000 7f743bfff640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 msgr2=0x7f742c0713b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/383804209 2024-09-10T18:53:57.065 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.030+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7f74300ae930 con 0x7f743c0fb800 2024-09-10T18:53:57.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.030+0000 7f743bfff640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 0x7f742c0713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:53:57.066 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.034+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7430077750 con 0x7f743c0fb800 2024-09-10T18:53:57.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.231+0000 7f743bfff640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 msgr2=0x7f742c0713b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/383804209 2024-09-10T18:53:57.232 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.231+0000 7f743bfff640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 0x7f742c0713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-09-10T18:53:57.241 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.240+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f743c10aa00 con 0x7f743c0fb800 2024-09-10T18:53:57.244 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.243+0000 7f74397fa640 1 -- 172.21.15.107:0/1948632480 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) v1 ==== 74+0+176735 (secure 0 0 0) 0x7f743007af60 con 0x7f743c0fb800 2024-09-10T18:53:57.247 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:53:57.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 msgr2=0x7f742c0713b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:53:57.248 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 0x7f742c0713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 msgr2=0x7f743c114d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f743002d040 tx=0x7f7430035040 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 shutdown_connections 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7f742c06ef00 0x7f742c0713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f743c1017d0 0x7f743c10ef20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f743c0fc1b0 0x7f743c1152d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 --2- 172.21.15.107:0/1948632480 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f743c0fb800 0x7f743c114d90 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.246+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 >> 172.21.15.107:0/1948632480 conn(0x7f743c071c00 msgr2=0x7f743c0f9f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.247+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 shutdown_connections 2024-09-10T18:53:57.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:53:57.247+0000 7f7442d2d640 1 -- 172.21.15.107:0/1948632480 wait complete. 2024-09-10T18:53:57.298 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:57.040+0000 7f598241fec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:53:57.323 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:57.055+0000 7f731a6eeec0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[29404]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[29404]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1948632480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:57.321+0000 7f731a6eeec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:57.382+0000 7f731a6eeec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:53:57.625 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:57.549+0000 7f731a6eeec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:53:57.626 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[40355]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:53:57.626 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[40355]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:53:57.626 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:53:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1948632480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:53:57.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-mon[33228]: from='mgr.14148 172.21.15.107:0/214390358' entity='mgr.y' cmd='[{"prefix": "mgr module enable", "module": "prometheus"}]': finished 2024-09-10T18:53:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-mon[33228]: mgrmap e17: y(active, since 4m), standbys: x 2024-09-10T18:53:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1948632480' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-09-10T18:53:57.674 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:57.297+0000 7f598241fec0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-09-10T18:53:57.674 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:57.356+0000 7f598241fec0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-09-10T18:53:57.674 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:57.517+0000 7f598241fec0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-09-10T18:53:57.909 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":17,"flags":0,"active_gid":14148,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6800","nonce":383804209},{"type":"v1","addr":"172.21.15.107:6801","nonce":383804209}]},"active_addr":"172.21.15.107:6801/383804209","active_change":"2024-09-10T18:49:03.110498+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":24106,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","prometheus","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.107:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"cephadm","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2976285793}]},{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":179573313}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2751636371}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":2923997335}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.107:0","nonce":1641515705}]}]} 2024-09-10T18:53:57.914 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-10T18:53:57.914 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-10T18:53:57.914 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:53:58.206 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:53:58.585 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:58.435+0000 7f731a6eeec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:53:58.585 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:58 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:58.500+0000 7f731a6eeec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:53:58.673 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:58.362+0000 7f598241fec0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-09-10T18:53:58.674 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:58.426+0000 7f598241fec0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-09-10T18:53:59.423 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.132+0000 7f598241fec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:53:59.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.244+0000 7f598241fec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:53:59.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.364+0000 7f598241fec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:53:59.450 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.217+0000 7f731a6eeec0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-09-10T18:53:59.450 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.328+0000 7f731a6eeec0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-09-10T18:53:59.675 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.424+0000 7f598241fec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:53:59.676 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.549+0000 7f598241fec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:53:59.676 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.617+0000 7f598241fec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:53:59.676 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.674+0000 7f598241fec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:53:59.709 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.448+0000 7f731a6eeec0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-09-10T18:53:59.709 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.508+0000 7f731a6eeec0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-09-10T18:53:59.709 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.635+0000 7f731a6eeec0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-09-10T18:53:59.709 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.707+0000 7f731a6eeec0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-09-10T18:53:59.934 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.807+0000 7f598241fec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:53:59.934 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.870+0000 7f598241fec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:53:59.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.765+0000 7f731a6eeec0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-09-10T18:53:59.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.901+0000 7f731a6eeec0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-09-10T18:54:00.237 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:53:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:53:59.964+0000 7f731a6eeec0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-09-10T18:54:00.237 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:00.035+0000 7f731a6eeec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:54:00.237 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:00.118+0000 7f731a6eeec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:54:00.237 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:00.175+0000 7f731a6eeec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:54:00.237 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:00.235+0000 7f731a6eeec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 -- 172.21.15.107:0/2693677475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c0fca50 msgr2=0x7fc03c0fcec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 --2- 172.21.15.107:0/2693677475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c0fca50 0x7fc03c0fcec0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc038004980 tx=0x7fc038009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 -- 172.21.15.107:0/2693677475 shutdown_connections 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 --2- 172.21.15.107:0/2693677475 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 --2- 172.21.15.107:0/2693677475 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c0fca50 0x7fc03c0fcec0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 --2- 172.21.15.107:0/2693677475 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c1028e0 0x7fc03c102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 -- 172.21.15.107:0/2693677475 >> 172.21.15.107:0/2693677475 conn(0x7fc03c073600 msgr2=0x7fc03c073a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 -- 172.21.15.107:0/2693677475 shutdown_connections 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.256+0000 7fc044796640 1 -- 172.21.15.107:0/2693677475 wait complete. 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 Processor -- start 2024-09-10T18:54:00.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 -- start start 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c0fca50 0x7fc03c112c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 0x7fc03c10cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10d400 con 0x7fc03c1028e0 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10d570 con 0x7fc03c0fca50 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc044796640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc03c10d730 con 0x7fc03c0fd400 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc041d31640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc041d31640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:38794/0 (socket says 172.21.15.107:38794) 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc041d31640 1 -- 172.21.15.107:0/3883518133 learned_addr learned my addr 172.21.15.107:0/3883518133 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.257+0000 7fc042d33640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 0x7fc03c10cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc042532640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c0fca50 0x7fc03c112c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 -- 172.21.15.107:0/3883518133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c0fca50 msgr2=0x7fc03c112c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:00.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c0fca50 0x7fc03c112c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 -- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 msgr2=0x7fc03c10cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 0x7fc03c10cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 -- 172.21.15.107:0/3883518133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc038005040 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc042d33640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 0x7fc03c10cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc041d31640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc03802a4c0 tx=0x7fc038035040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc038007d00 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fc038033420 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc03803c410 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc03c10d9b0 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.258+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc03c119c10 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.259+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc03c0fe000 con 0x7fc03c0fd400 2024-09-10T18:54:00.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.260+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 17) v1 ==== 90846+0+0 (secure 0 0 0) 0x7fc038030020 con 0x7fc03c0fd400 2024-09-10T18:54:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.260+0000 7fc032ffd640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 0x7fc0280713b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.260+0000 7fc042532640 1 -- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 msgr2=0x7fc0280713b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.107:6800/383804209 2024-09-10T18:54:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.260+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 7144+0+0 (secure 0 0 0) 0x7fc0380ae870 con 0x7fc03c0fd400 2024-09-10T18:54:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.260+0000 7fc042532640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 0x7fc0280713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-09-10T18:54:00.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.264+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fc0380775d0 con 0x7fc03c0fd400 2024-09-10T18:54:00.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:53:59 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:53:59.933+0000 7f598241fec0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-09-10T18:54:00.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:54:00.016+0000 7f598241fec0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-09-10T18:54:00.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:54:00.074+0000 7f598241fec0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-09-10T18:54:00.424 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:54:00.131+0000 7f598241fec0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-09-10T18:54:00.440 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.439+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc03c10e2c0 con 0x7fc03c0fd400 2024-09-10T18:54:00.441 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.440+0000 7fc032ffd640 1 -- 172.21.15.107:0/3883518133 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+21000 (secure 0 0 0) 0x7fc03807ade0 con 0x7fc03c0fd400 2024-09-10T18:54:00.441 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:54:00.442 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":60,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","created":"2024-09-10T18:48:17.431796+0000","modified":"2024-09-10T18:53:37.142440+0000","last_up_change":"2024-09-10T18:53:10.945089+0000","last_in_change":"2024-09-10T18:52:53.992364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:51:13.408728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:53:22.770011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:53:23.112734+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:53:24.591420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:53:26.539840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:53:28.607061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"1106f881-aeec-499d-9e94-6d0e2a28a6d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1943775658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1943775658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6809","nonce":1943775658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1943775658}]},"public_addr":"172.21.15.107:6803/1943775658","cluster_addr":"172.21.15.107:6805/1943775658","heartbeat_back_addr":"172.21.15.107:6809/1943775658","heartbeat_front_addr":"172.21.15.107:6807/1943775658","state":["exists","up"]},{"osd":1,"uuid":"7c834ad4-a6e2-408c-87fa-2953804ec22c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6811","nonce":462447353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6813","nonce":462447353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6817","nonce":462447353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6815","nonce":462447353}]},"public_addr":"172.21.15.107:6811/462447353","cluster_addr":"172.21.15.107:6813/462447353","heartbeat_back_addr":"172.21.15.107:6817/462447353","heartbeat_front_addr":"172.21.15.107:6815/462447353","state":["exists","up"]},{"osd":2,"uuid":"c61bf972-04c3-4f95-b23f-89662ed8c66d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6819","nonce":1138865377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6821","nonce":1138865377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6825","nonce":1138865377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6823","nonce":1138865377}]},"public_addr":"172.21.15.107:6819/1138865377","cluster_addr":"172.21.15.107:6821/1138865377","heartbeat_back_addr":"172.21.15.107:6825/1138865377","heartbeat_front_addr":"172.21.15.107:6823/1138865377","state":["exists","up"]},{"osd":3,"uuid":"b42bf6e2-ce5b-446e-82e7-ea673401b1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6827","nonce":3842970087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6829","nonce":3842970087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6833","nonce":3842970087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6831","nonce":3842970087}]},"public_addr":"172.21.15.107:6827/3842970087","cluster_addr":"172.21.15.107:6829/3842970087","heartbeat_back_addr":"172.21.15.107:6833/3842970087","heartbeat_front_addr":"172.21.15.107:6831/3842970087","state":["exists","up"]},{"osd":4,"uuid":"6de68f17-c6b5-4b80-8a69-e27860c86651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6801","nonce":335919521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6803","nonce":335919521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6807","nonce":335919521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6805","nonce":335919521}]},"public_addr":"172.21.15.162:6801/335919521","cluster_addr":"172.21.15.162:6803/335919521","heartbeat_back_addr":"172.21.15.162:6807/335919521","heartbeat_front_addr":"172.21.15.162:6805/335919521","state":["exists","up"]},{"osd":5,"uuid":"88d7424d-a81a-4695-8f1e-cda6b4bc228c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6809","nonce":688086025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6811","nonce":688086025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6815","nonce":688086025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6813","nonce":688086025}]},"public_addr":"172.21.15.162:6809/688086025","cluster_addr":"172.21.15.162:6811/688086025","heartbeat_back_addr":"172.21.15.162:6815/688086025","heartbeat_front_addr":"172.21.15.162:6813/688086025","state":["exists","up"]},{"osd":6,"uuid":"ccaefb3e-2fdf-4af2-a846-6ed2dcae1119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6817","nonce":1609004877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6819","nonce":1609004877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6823","nonce":1609004877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6821","nonce":1609004877}]},"public_addr":"172.21.15.162:6817/1609004877","cluster_addr":"172.21.15.162:6819/1609004877","heartbeat_back_addr":"172.21.15.162:6823/1609004877","heartbeat_front_addr":"172.21.15.162:6821/1609004877","state":["exists","up"]},{"osd":7,"uuid":"9af4b3a1-c67b-4900-bf78-7c93e7efe0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6825","nonce":3888567684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6827","nonce":3888567684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6831","nonce":3888567684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6829","nonce":3888567684}]},"public_addr":"172.21.15.162:6825/3888567684","cluster_addr":"172.21.15.162:6827/3888567684","heartbeat_back_addr":"172.21.15.162:6831/3888567684","heartbeat_front_addr":"172.21.15.162:6829/3888567684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:23.427116+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:46.698818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:10.767915+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:34.742041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:56.496900+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:20.395868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:44.954153+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:53:09.043479+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1327904374":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6801/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/3160890405":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/46900690":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/72140394":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/2982976703":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/159250052":"2024-09-11T18:48:40.231412+0000","172.21.15.107:6801/1592664852":"2024-09-11T18:48:40.231412+0000","172.21.15.107:6800/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6800/1592664852":"2024-09-11T18:48:40.231412+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":2,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:54:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 msgr2=0x7fc0280713b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:54:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 0x7fc0280713b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.443 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 msgr2=0x7fc03c113160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fc03802a4c0 tx=0x7fc038035040 comp rx=0 tx=0).stop 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 shutdown_connections 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:6800/383804209,v1:172.21.15.107:6801/383804209] conn(0x7fc02806ef00 0x7fc0280713b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fc03c1028e0 0x7fc03c10cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fc03c0fd400 0x7fc03c113160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 --2- 172.21.15.107:0/3883518133 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fc03c0fca50 0x7fc03c112c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 >> 172.21.15.107:0/3883518133 conn(0x7fc03c073600 msgr2=0x7fc03c0fb850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 shutdown_connections 2024-09-10T18:54:00.444 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:00.442+0000 7fc044796640 1 -- 172.21.15.107:0/3883518133 wait complete. 2024-09-10T18:54:00.808 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3883518133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:00.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3883518133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:00.879 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3883518133' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:01.076 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:01.015+0000 7f731a6eeec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:54:00.878+0000 7f598241fec0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: [10/Sep/2024:18:54:00] ENGINE Bus STARTING 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: CherryPy Checker: 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: The Application mounted at '' has an empty config. 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: [10/Sep/2024:18:54:00] ENGINE Serving on http://:::9283 2024-09-10T18:54:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: [10/Sep/2024:18:54:00] ENGINE Bus STARTED 2024-09-10T18:54:01.219 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-10T18:54:01.219 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd dump --format=json 2024-09-10T18:54:01.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:01] ENGINE Bus STARTING 2024-09-10T18:54:01.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: CherryPy Checker: 2024-09-10T18:54:01.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: The Application mounted at '' has an empty config. 2024-09-10T18:54:01.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:01.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:01] ENGINE Serving on http://:::9283 2024-09-10T18:54:01.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:01] ENGINE Bus STARTED 2024-09-10T18:54:01.485 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: Standby manager daemon x restarted 2024-09-10T18:54:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: Standby manager daemon x started 2024-09-10T18:54:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:54:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:54:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: Active manager daemon y restarted 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: Activating manager daemon y 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: mgrmap e18: y(active, starting, since 0.00725775s), standbys: x 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:54:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: Manager daemon y is now available 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:54:01.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:54:01.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: Standby manager daemon x restarted 2024-09-10T18:54:01.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: Standby manager daemon x started 2024-09-10T18:54:01.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:54:01.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:54:01.750 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: Active manager daemon y restarted 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: Activating manager daemon y 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: mgrmap e18: y(active, starting, since 0.00725775s), standbys: x 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:54:01.751 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: Manager daemon y is now available 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:01.752 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: Standby manager daemon x restarted 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: Standby manager daemon x started 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/crt"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/x/key"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.? 172.21.15.162:0/1397506685' entity='mgr.x' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: Active manager daemon y restarted 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: Activating manager daemon y 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: osdmap e61: 8 total, 8 up, 8 in 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: mgrmap e18: y(active, starting, since 0.00725775s), standbys: x 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-09-10T18:54:01.753 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-09-10T18:54:01.754 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: Manager daemon y is now available 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/mirror_snapshot_schedule"}]: dispatch 2024-09-10T18:54:01.755 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/y/trash_purge_schedule"}]: dispatch 2024-09-10T18:54:02.362 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:02.462+0000 7f72bd470640 -1 log_channel(cephadm) log [ERR] : [10/Sep/2024:18:54:02] ENGINE Error in HTTPServer.serve 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: Traceback (most recent call last): 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._connections.run(self.expiration_interval) 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:54:02.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._run(expiration_interval) 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: s = self.context.wrap_socket( 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: return self.sslsocket_class._create( 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self.do_handshake() 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: self._sslobj.do_handshake() 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:54:02.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:02 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: mgrmap e19: y(active, since 1.01142s), standbys: x 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: [10/Sep/2024:18:54:02] ENGINE Bus STARTING 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: [10/Sep/2024:18:54:02] ENGINE Serving on https://172.21.15.107:7150 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: [10/Sep/2024:18:54:02] ENGINE Bus STARTED 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: [10/Sep/2024:18:54:02] ENGINE Error in HTTPServer.serve 2024-09-10T18:54:03.094 INFO:journalctl@ceph.mon.a.smithi107.stdout: Traceback (most recent call last): 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._run(expiration_interval) 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: s = self.context.wrap_socket( 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: return self.sslsocket_class._create( 2024-09-10T18:54:03.095 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.a.smithi107.stdout: self.do_handshake() 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.a.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.a.smithi107.stdout: self._sslobj.do_handshake() 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.a.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: mgrmap e19: y(active, since 1.01142s), standbys: x 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: [10/Sep/2024:18:54:02] ENGINE Bus STARTING 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: [10/Sep/2024:18:54:02] ENGINE Serving on https://172.21.15.107:7150 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: [10/Sep/2024:18:54:02] ENGINE Bus STARTED 2024-09-10T18:54:03.096 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: [10/Sep/2024:18:54:02] ENGINE Error in HTTPServer.serve 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: Traceback (most recent call last): 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._run(expiration_interval) 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:54:03.097 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: s = self.context.wrap_socket( 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: return self.sslsocket_class._create( 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: self.do_handshake() 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: self._sslobj.do_handshake() 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:54:03.098 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:03 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.323+0000 7f229d518640 1 -- 172.21.15.107:0/2428097633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 msgr2=0x7f2298108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.323+0000 7f229d518640 1 --2- 172.21.15.107:0/2428097633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298108580 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2290009d00 tx=0x7f2290004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- 172.21.15.107:0/2428097633 shutdown_connections 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- 172.21.15.107:0/2428097633 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298108580 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- 172.21.15.107:0/2428097633 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 0x7f22980fbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- 172.21.15.107:0/2428097633 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 0x7f2298101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.337 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- 172.21.15.107:0/2428097633 >> 172.21.15.107:0/2428097633 conn(0x7f2298071c00 msgr2=0x7f2298072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- 172.21.15.107:0/2428097633 shutdown_connections 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- 172.21.15.107:0/2428097633 wait complete. 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 Processor -- start 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- start start 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 0x7f22981118b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 0x7f229810ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.324+0000 7f229d518640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229810c0b0 con 0x7f22980fc1b0 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f229d518640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f229810c220 con 0x7f22980fb800 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f229d518640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f229810c3e0 con 0x7f22981017d0 2024-09-10T18:54:03.338 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f22977fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 0x7f229810ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:34992/0 (socket says 172.21.15.107:34992) 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 -- 172.21.15.107:0/1940968112 learned_addr learned my addr 172.21.15.107:0/1940968112 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f2296ffd640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 0x7f22981118b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 -- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 msgr2=0x7f229810ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 0x7f229810ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 -- 172.21.15.107:0/1940968112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 msgr2=0x7f22981118b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 0x7f22981118b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 -- 172.21.15.107:0/1940968112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288005040 con 0x7f22980fc1b0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228e7fc640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2280002ad0 tx=0x7f2280004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280007c30 con 0x7f22980fc1b0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22900098b0 con 0x7f22980fc1b0 2024-09-10T18:54:03.339 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f228000f040 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2280009340 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.325+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2298118750 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.326+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22980fcef0 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.327+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 19) v1 ==== 90589+0+0 (secure 0 0 0) 0x7f22800096f0 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.327+0000 7f228ffff640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 0x7f22840710c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.327+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f228008e330 con 0x7f22980fc1b0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.327+0000 7f2296ffd640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 0x7f22840710c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.328+0000 7f2296ffd640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 0x7f22840710c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2288004a50 tx=0x7f228800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:03.340 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.331+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f22800579f0 con 0x7f22980fc1b0 2024-09-10T18:54:03.341 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: mgrmap e19: y(active, since 1.01142s), standbys: x 2024-09-10T18:54:03.341 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: [10/Sep/2024:18:54:02] ENGINE Bus STARTING 2024-09-10T18:54:03.341 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: [10/Sep/2024:18:54:02] ENGINE Serving on https://172.21.15.107:7150 2024-09-10T18:54:03.341 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: [10/Sep/2024:18:54:02] ENGINE Bus STARTED 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: [10/Sep/2024:18:54:02] ENGINE Error in HTTPServer.serve 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: Traceback (most recent call last): 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: self._connections.run(self.expiration_interval) 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: self._run(expiration_interval) 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: s = self.context.wrap_socket( 2024-09-10T18:54:03.342 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: return self.sslsocket_class._create( 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: self.do_handshake() 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: self._sslobj.do_handshake() 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-10T18:54:03.343 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:03 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:03.472 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.468+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mgrmap(e 20) v1 ==== 90783+0+0 (secure 0 0 0) 0x7f2280057630 con 0x7f22980fc1b0 2024-09-10T18:54:03.502 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.501+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f229810cf20 con 0x7f22980fc1b0 2024-09-10T18:54:03.503 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.501+0000 7f228ffff640 1 -- 172.21.15.107:0/1940968112 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+21401 (secure 0 0 0) 0x7f2280057260 con 0x7f22980fc1b0 2024-09-10T18:54:03.503 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:54:03.503 INFO:teuthology.orchestra.run.smithi107.stdout:{"epoch":61,"fsid":"d7c9fa7c-6fa4-11ef-bcea-c7b262605968","created":"2024-09-10T18:48:17.431796+0000","modified":"2024-09-10T18:54:01.018028+0000","last_up_change":"2024-09-10T18:53:10.945089+0000","last_in_change":"2024-09-10T18:52:53.992364+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":18,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-10T18:51:13.408728+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":".rgw.root","create_time":"2024-09-10T18:53:22.770011+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"53","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":3,"pool_name":"datapool","create_time":"2024-09-10T18:53:23.112734+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":3,"pg_placement_num":3,"pg_placement_num_target":3,"pg_num_target":3,"pg_num_pending":3,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"56","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":56,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":4,"pool_name":"default.rgw.log","create_time":"2024-09-10T18:53:24.591420+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"55","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":5,"pool_name":"default.rgw.control","create_time":"2024-09-10T18:53:26.539840+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"57","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}}},{"pool":6,"pool_name":"default.rgw.meta","create_time":"2024-09-10T18:53:28.607061+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"59","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}}}],"osds":[{"osd":0,"uuid":"1106f881-aeec-499d-9e94-6d0e2a28a6d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6802","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6803","nonce":1943775658}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6804","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6805","nonce":1943775658}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6808","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6809","nonce":1943775658}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6806","nonce":1943775658},{"type":"v1","addr":"172.21.15.107:6807","nonce":1943775658}]},"public_addr":"172.21.15.107:6803/1943775658","cluster_addr":"172.21.15.107:6805/1943775658","heartbeat_back_addr":"172.21.15.107:6809/1943775658","heartbeat_front_addr":"172.21.15.107:6807/1943775658","state":["exists","up"]},{"osd":1,"uuid":"7c834ad4-a6e2-408c-87fa-2953804ec22c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6810","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6811","nonce":462447353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6812","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6813","nonce":462447353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6816","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6817","nonce":462447353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6814","nonce":462447353},{"type":"v1","addr":"172.21.15.107:6815","nonce":462447353}]},"public_addr":"172.21.15.107:6811/462447353","cluster_addr":"172.21.15.107:6813/462447353","heartbeat_back_addr":"172.21.15.107:6817/462447353","heartbeat_front_addr":"172.21.15.107:6815/462447353","state":["exists","up"]},{"osd":2,"uuid":"c61bf972-04c3-4f95-b23f-89662ed8c66d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6818","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6819","nonce":1138865377}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6820","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6821","nonce":1138865377}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6824","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6825","nonce":1138865377}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6822","nonce":1138865377},{"type":"v1","addr":"172.21.15.107:6823","nonce":1138865377}]},"public_addr":"172.21.15.107:6819/1138865377","cluster_addr":"172.21.15.107:6821/1138865377","heartbeat_back_addr":"172.21.15.107:6825/1138865377","heartbeat_front_addr":"172.21.15.107:6823/1138865377","state":["exists","up"]},{"osd":3,"uuid":"b42bf6e2-ce5b-446e-82e7-ea673401b1bb","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6826","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6827","nonce":3842970087}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6828","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6829","nonce":3842970087}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6832","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6833","nonce":3842970087}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.107:6830","nonce":3842970087},{"type":"v1","addr":"172.21.15.107:6831","nonce":3842970087}]},"public_addr":"172.21.15.107:6827/3842970087","cluster_addr":"172.21.15.107:6829/3842970087","heartbeat_back_addr":"172.21.15.107:6833/3842970087","heartbeat_front_addr":"172.21.15.107:6831/3842970087","state":["exists","up"]},{"osd":4,"uuid":"6de68f17-c6b5-4b80-8a69-e27860c86651","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":30,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6800","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6801","nonce":335919521}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6802","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6803","nonce":335919521}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6806","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6807","nonce":335919521}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6804","nonce":335919521},{"type":"v1","addr":"172.21.15.162:6805","nonce":335919521}]},"public_addr":"172.21.15.162:6801/335919521","cluster_addr":"172.21.15.162:6803/335919521","heartbeat_back_addr":"172.21.15.162:6807/335919521","heartbeat_front_addr":"172.21.15.162:6805/335919521","state":["exists","up"]},{"osd":5,"uuid":"88d7424d-a81a-4695-8f1e-cda6b4bc228c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6808","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6809","nonce":688086025}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6810","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6811","nonce":688086025}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6814","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6815","nonce":688086025}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6812","nonce":688086025},{"type":"v1","addr":"172.21.15.162:6813","nonce":688086025}]},"public_addr":"172.21.15.162:6809/688086025","cluster_addr":"172.21.15.162:6811/688086025","heartbeat_back_addr":"172.21.15.162:6815/688086025","heartbeat_front_addr":"172.21.15.162:6813/688086025","state":["exists","up"]},{"osd":6,"uuid":"ccaefb3e-2fdf-4af2-a846-6ed2dcae1119","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":55,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6816","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6817","nonce":1609004877}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6818","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6819","nonce":1609004877}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6822","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6823","nonce":1609004877}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6820","nonce":1609004877},{"type":"v1","addr":"172.21.15.162:6821","nonce":1609004877}]},"public_addr":"172.21.15.162:6817/1609004877","cluster_addr":"172.21.15.162:6819/1609004877","heartbeat_back_addr":"172.21.15.162:6823/1609004877","heartbeat_front_addr":"172.21.15.162:6821/1609004877","state":["exists","up"]},{"osd":7,"uuid":"9af4b3a1-c67b-4900-bf78-7c93e7efe0b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":57,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6824","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6825","nonce":3888567684}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6826","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6827","nonce":3888567684}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6830","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6831","nonce":3888567684}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.162:6828","nonce":3888567684},{"type":"v1","addr":"172.21.15.162:6829","nonce":3888567684}]},"public_addr":"172.21.15.162:6825/3888567684","cluster_addr":"172.21.15.162:6827/3888567684","heartbeat_back_addr":"172.21.15.162:6831/3888567684","heartbeat_front_addr":"172.21.15.162:6829/3888567684","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:23.427116+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:50:46.698818+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:10.767915+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:34.742041+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:51:56.496900+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:20.395868+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:52:44.954153+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2024-09-10T18:53:09.043479+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.107:0/1641515705":"2024-09-11T18:54:01.017991+0000","172.21.15.107:0/2923997335":"2024-09-11T18:54:01.017991+0000","172.21.15.107:0/2976285793":"2024-09-11T18:54:01.017991+0000","172.21.15.107:6800/1592664852":"2024-09-11T18:48:40.231412+0000","172.21.15.107:6801/1592664852":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/159250052":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/2982976703":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/72140394":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6801/383804209":"2024-09-11T18:54:01.017991+0000","172.21.15.107:0/2751636371":"2024-09-11T18:54:01.017991+0000","172.21.15.107:6800/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/46900690":"2024-09-11T18:48:40.231412+0000","172.21.15.107:0/179573313":"2024-09-11T18:54:01.017991+0000","172.21.15.107:0/3160890405":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6801/2559986261":"2024-09-11T18:49:03.110395+0000","172.21.15.107:0/1327904374":"2024-09-11T18:49:03.110395+0000","172.21.15.107:6800/383804209":"2024-09-11T18:54:01.017991+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-10T18:54:03.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.503+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 msgr2=0x7f22840710c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:03.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.503+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 0x7f22840710c0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2288004a50 tx=0x7f228800b040 comp rx=0 tx=0).stop 2024-09-10T18:54:03.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.503+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 msgr2=0x7f2298111df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:03.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.503+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f2280002ad0 tx=0x7f2280004600 comp rx=0 tx=0).stop 2024-09-10T18:54:03.505 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 shutdown_connections 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f228406ec10 0x7f22840710c0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f22981017d0 0x7f229810ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f22980fc1b0 0x7f2298111df0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 --2- 172.21.15.107:0/1940968112 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f22980fb800 0x7f22981118b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 >> 172.21.15.107:0/1940968112 conn(0x7f2298071c00 msgr2=0x7f22980f9e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 shutdown_connections 2024-09-10T18:54:03.506 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:03.504+0000 7f229d518640 1 -- 172.21.15.107:0/1940968112 wait complete. 2024-09-10T18:54:04.495 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-09-10T18:54:04.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-09-10T18:54:04.496 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-09-10T18:54:04.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-09-10T18:54:04.497 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-09-10T18:54:04.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-09-10T18:54:04.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-09-10T18:54:04.498 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-09-10T18:54:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:04 smithi162 ceph-mon[33228]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:04 smithi162 ceph-mon[33228]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:54:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1940968112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:04 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:04 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[40355]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[40355]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1940968112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[29404]: pgmap v4: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[29404]: mgrmap e20: y(active, since 2s), standbys: x 2024-09-10T18:54:04.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1940968112' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-09-10T18:54:04.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:04 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:04.978 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.029 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.032 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.032 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.036 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.040 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.043 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:05.271 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 3760M 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[29404]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:06.349 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 3760M 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.350 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:05 smithi107 ceph-mon[40355]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 3760M 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:05 smithi162 ceph-mon[33228]: pgmap v5: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:07.339 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:07 smithi107 ceph-mon[29404]: mgrmap e21: y(active, since 5s), standbys: x 2024-09-10T18:54:07.339 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:07 smithi107 ceph-mon[40355]: mgrmap e21: y(active, since 5s), standbys: x 2024-09-10T18:54:07.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:07 smithi162 ceph-mon[33228]: mgrmap e21: y(active, since 5s), standbys: x 2024-09-10T18:54:08.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:08 smithi162 ceph-mon[33228]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:08 smithi107 ceph-mon[29404]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:08.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:08 smithi107 ceph-mon[40355]: pgmap v6: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:54:10.306 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:10 smithi107 ceph-mon[29404]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:54:10.306 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:10 smithi107 ceph-mon[40355]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:54:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:10 smithi162 ceph-mon[33228]: pgmap v7: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 26 KiB/s rd, 0 B/s wr, 11 op/s 2024-09-10T18:54:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:54:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 -- 172.21.15.107:0/2489938639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 msgr2=0x7f59e4108520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 --2- 172.21.15.107:0/2489938639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4108520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009d00 tx=0x7f59d4004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 -- 172.21.15.107:0/2489938639 shutdown_connections 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 --2- 172.21.15.107:0/2489938639 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4108520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 --2- 172.21.15.107:0/2489938639 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e40fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 --2- 172.21.15.107:0/2489938639 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e41017b0 0x7f59e4101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 -- 172.21.15.107:0/2489938639 >> 172.21.15.107:0/2489938639 conn(0x7f59e40724e0 msgr2=0x7f59e40728e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 -- 172.21.15.107:0/2489938639 shutdown_connections 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.510+0000 7f59eb5df640 1 -- 172.21.15.107:0/2489938639 wait complete. 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 Processor -- start 2024-09-10T18:54:11.539 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 -- start start 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e4111970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4111eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e41017b0 0x7f59e410bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e410c170 con 0x7f59e40fc2e0 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59e410c2e0 con 0x7f59e41017b0 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59eb5df640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59e410c4a0 con 0x7f59e40fb950 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59e937b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e4111970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:11.540 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59e8b7a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4111eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59e937b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e4111970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:36632/0 (socket says 172.21.15.107:36632) 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59e937b640 1 -- 172.21.15.107:0/549324144 learned_addr learned my addr 172.21.15.107:0/549324144 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.511+0000 7f59e8b7a640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4111eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:51786/0 (socket says 172.21.15.107:51786) 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e9b7c640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e41017b0 0x7f59e410bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 -- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e41017b0 msgr2=0x7f59e410bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f59e41017b0 0x7f59e410bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 -- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 msgr2=0x7f59e4111eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4111eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.541 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59d8005040 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e8b7a640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f59e40fc2e0 0x7f59e4111eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e937b640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e4111970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00db70 tx=0x7f59dc00b460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc00ec30 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59d40098b0 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f59dc00ed90 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59dc016640 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.512+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59e4065fa0 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.513+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f59e4100c40 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f59dc014070 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f59cc06ef10 0x7f59cc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f59dc090a90 con 0x7f59e40fb950 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e8b7a640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f59cc06ef10 0x7f59cc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:11.542 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] conn(0x7f59cc078c40 0x7f59cc07b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f59cc07b740 con 0x7f59cc078c40 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.514+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f59dc095240 con 0x7f59e40fb950 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.515+0000 7f59e8b7a640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f59cc06ef10 0x7f59cc0713c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59e410ce00 tx=0x7f59d800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.515+0000 7f59e9b7c640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] conn(0x7f59cc078c40 0x7f59cc07b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.515+0000 7f59e9b7c640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] conn(0x7f59cc078c40 0x7f59cc07b070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.516+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== osd.6 v2:172.21.15.162:6816/1609004877 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f59cc07b740 con 0x7f59cc078c40 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.528+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 --> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f59e4100ff0 con 0x7f59cc078c40 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59e1ffb640 1 -- 172.21.15.107:0/549324144 <== osd.6 v2:172.21.15.162:6816/1609004877 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f59e4100ff0 con 0x7f59cc078c40 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] conn(0x7f59cc078c40 msgr2=0x7f59cc07b070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.162:6816/1609004877,v1:172.21.15.162:6817/1609004877] conn(0x7f59cc078c40 0x7f59cc07b070 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f59cc06ef10 msgr2=0x7f59cc0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f59cc06ef10 0x7f59cc0713c0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f59e410ce00 tx=0x7f59d800b040 comp rx=0 tx=0).stop 2024-09-10T18:54:11.543 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 msgr2=0x7f59e4111970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 --2- 172.21.15.107:0/549324144 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f59e40fb950 0x7f59e4111970 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f59dc00db70 tx=0x7f59dc00b460 comp rx=0 tx=0).stop 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59e9b7c640 1 -- 172.21.15.107:0/549324144 reap_dead start 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 shutdown_connections 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 >> 172.21.15.107:0/549324144 conn(0x7f59e40724e0 msgr2=0x7f59e40ff2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 shutdown_connections 2024-09-10T18:54:11.544 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:11.529+0000 7f59eb5df640 1 -- 172.21.15.107:0/549324144 wait complete. 2024-09-10T18:54:12.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:12 smithi107 ceph-mon[29404]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:54:12.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:12 smithi107 ceph-mon[40355]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:54:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:12 smithi162 ceph-mon[33228]: pgmap v8: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 20 KiB/s rd, 0 B/s wr, 8 op/s 2024-09-10T18:54:12.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:13.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 -- 172.21.15.107:0/2948409336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fe250 msgr2=0x7f55b8106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 --2- 172.21.15.107:0/2948409336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fe250 0x7f55b8106890 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f55ac004890 tx=0x7f55ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 -- 172.21.15.107:0/2948409336 shutdown_connections 2024-09-10T18:54:13.118 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 --2- 172.21.15.107:0/2948409336 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b8106e60 0x7f55b8109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 --2- 172.21.15.107:0/2948409336 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fe250 0x7f55b8106890 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 --2- 172.21.15.107:0/2948409336 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b80fd8b0 0x7f55b80fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 -- 172.21.15.107:0/2948409336 >> 172.21.15.107:0/2948409336 conn(0x7f55b8071c00 msgr2=0x7f55b8072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 -- 172.21.15.107:0/2948409336 shutdown_connections 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.010+0000 7f55be528640 1 -- 172.21.15.107:0/2948409336 wait complete. 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 Processor -- start 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 -- start start 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 0x7f55b8112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80fe250 0x7f55b81131f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b8106e60 0x7f55b810ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b810d490 con 0x7f55b8106e60 2024-09-10T18:54:13.119 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55b810d600 con 0x7f55b80fe250 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55be528640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55b810d7c0 con 0x7f55b80fd8b0 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 0x7f55b8112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55bcac5640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b8106e60 0x7f55b810ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 0x7f55b8112cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:36652/0 (socket says 172.21.15.107:36652) 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 -- 172.21.15.107:0/4051715 learned_addr learned my addr 172.21.15.107:0/4051715 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:13.120 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b77fe640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80fe250 0x7f55b81131f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 -- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80fe250 msgr2=0x7f55b81131f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f55b80fe250 0x7f55b81131f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 -- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b8106e60 msgr2=0x7f55b810ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f55b8106e60 0x7f55b810ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.011+0000 7f55b7fff640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55a8005040 con 0x7f55b80fd8b0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55b7fff640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 0x7f55b8112cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009550 tx=0x7f55a8009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a800e930 con 0x7f55b80fd8b0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55ac005040 con 0x7f55b80fd8b0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f55a800f040 con 0x7f55b80fd8b0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55a8013af0 con 0x7f55b80fd8b0 2024-09-10T18:54:13.121 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.012+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55b8119b00 con 0x7f55b80fd8b0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.013+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f55b805dd90 con 0x7f55b80fd8b0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.013+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f55a800ea90 con 0x7f55b80fd8b0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b4ff9640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f55a006ef10 0x7f55a00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f55a808e780 con 0x7f55b80fd8b0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b77fe640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f55a006ef10 0x7f55a00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b4ff9640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] conn(0x7f55a0078d10 0x7f55a007b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f55a007b810 con 0x7f55a0078d10 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f55a8090dc0 con 0x7f55b80fd8b0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55b77fe640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f55a006ef10 0x7f55a00713c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009040 tx=0x7f55ac009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.014+0000 7f55bcac5640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] conn(0x7f55a0078d10 0x7f55a007b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.015+0000 7f55bcac5640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] conn(0x7f55a0078d10 0x7f55a007b140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.015+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== osd.4 v2:172.21.15.162:6800/335919521 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f55a007b810 con 0x7f55a0078d10 2024-09-10T18:54:13.122 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.029+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 --> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f55b8065b20 con 0x7f55a0078d10 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.029+0000 7f55b4ff9640 1 -- 172.21.15.107:0/4051715 <== osd.4 v2:172.21.15.162:6800/335919521 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f55b8065b20 con 0x7f55a0078d10 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.029+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] conn(0x7f55a0078d10 msgr2=0x7f55a007b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.029+0000 7f55be528640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.162:6800/335919521,v1:172.21.15.162:6801/335919521] conn(0x7f55a0078d10 0x7f55a007b140 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f55a006ef10 msgr2=0x7f55a00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f55a006ef10 0x7f55a00713c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f55ac009040 tx=0x7f55ac009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 msgr2=0x7f55b8112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 --2- 172.21.15.107:0/4051715 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f55b80fd8b0 0x7f55b8112cb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f55a8009550 tx=0x7f55a8009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55bcac5640 1 -- 172.21.15.107:0/4051715 reap_dead start 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 shutdown_connections 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 >> 172.21.15.107:0/4051715 conn(0x7f55b8071c00 msgr2=0x7f55b8107610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 shutdown_connections 2024-09-10T18:54:13.123 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.030+0000 7f55be528640 1 -- 172.21.15.107:0/4051715 wait complete. 2024-09-10T18:54:13.342 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:13.430 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:13.430 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:14.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2038871525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0fe410 msgr2=0x7f8a9c0fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2038871525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0fe410 0x7f8a9c0fe860 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009d00 tx=0x7f8a90004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:14.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2038871525 shutdown_connections 2024-09-10T18:54:14.023 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2038871525 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a9c0feda0 0x7f8a9c1097a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2038871525 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0fe410 0x7f8a9c0fe860 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2038871525 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fcc90 0x7f8a9c0fd060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.097+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2038871525 >> 172.21.15.107:0/2038871525 conn(0x7f8a9c072550 msgr2=0x7f8a9c0faad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2038871525 shutdown_connections 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2038871525 wait complete. 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 Processor -- start 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- start start 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a9c0fcc90 0x7f8a9c112c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 0x7f8a9c1131d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0feda0 0x7f8a9c10ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:14.024 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c10d470 con 0x7f8a9c0feda0 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c10d5e0 con 0x7f8a9c0fcc90 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa44b6640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a9c10d7a0 con 0x7f8a9c0fe410 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa1a51640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 0x7f8a9c1131d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa2a53640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0feda0 0x7f8a9c10ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa1a51640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 0x7f8a9c1131d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:36664/0 (socket says 172.21.15.107:36664) 2024-09-10T18:54:14.025 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa1a51640 1 -- 172.21.15.107:0/2208196904 learned_addr learned my addr 172.21.15.107:0/2208196904 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.098+0000 7f8aa2252640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a9c0fcc90 0x7f8a9c112c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 -- 172.21.15.107:0/2208196904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a9c0fcc90 msgr2=0x7f8a9c112c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8a9c0fcc90 0x7f8a9c112c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 -- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0feda0 msgr2=0x7f8a9c10ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0feda0 0x7f8a9c10ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a900098b0 con 0x7f8a9c0fe410 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa2a53640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8a9c0feda0 0x7f8a9c10ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa1a51640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 0x7f8a9c1131d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a90033920 tx=0x7f8a90004d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a90038070 con 0x7f8a9c0fe410 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8a90033de0 con 0x7f8a9c0fe410 2024-09-10T18:54:14.026 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a9c10da20 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a9003c720 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.099+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a9c119c80 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.100+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8a9c05ddd0 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.100+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f8a90030070 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8a9affd640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8a8406ef10 0x7f8a840713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8aa2252640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8a8406ef10 0x7f8a840713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f8a900aeb30 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8a9affd640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] conn(0x7f8a84078d90 0x7f8a8407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8a8407b890 con 0x7f8a84078d90 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8a900b3240 con 0x7f8a9c0fe410 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8aa2a53640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] conn(0x7f8a84078d90 0x7f8a8407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:14.027 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.101+0000 7f8aa2252640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8a8406ef10 0x7f8a840713c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d70 tx=0x7f8a9400b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.102+0000 7f8aa2a53640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] conn(0x7f8a84078d90 0x7f8a8407b1c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.102+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== osd.0 v2:172.21.15.107:6802/1943775658 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8a8407b890 con 0x7f8a84078d90 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.114+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 --> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8a9c05e180 con 0x7f8a84078d90 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.114+0000 7f8a9affd640 1 -- 172.21.15.107:0/2208196904 <== osd.0 v2:172.21.15.107:6802/1943775658 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f8a9c05e180 con 0x7f8a84078d90 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] conn(0x7f8a84078d90 msgr2=0x7f8a8407b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6802/1943775658,v1:172.21.15.107:6803/1943775658] conn(0x7f8a84078d90 0x7f8a8407b1c0 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8a8406ef10 msgr2=0x7f8a840713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8a8406ef10 0x7f8a840713c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8a94009d70 tx=0x7f8a9400b040 comp rx=0 tx=0).stop 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 msgr2=0x7f8a9c1131d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 --2- 172.21.15.107:0/2208196904 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8a9c0fe410 0x7f8a9c1131d0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8a90033920 tx=0x7f8a90004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa2a53640 1 -- 172.21.15.107:0/2208196904 reap_dead start 2024-09-10T18:54:14.028 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 shutdown_connections 2024-09-10T18:54:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 >> 172.21.15.107:0/2208196904 conn(0x7f8a9c072550 msgr2=0x7f8a9c072c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 shutdown_connections 2024-09-10T18:54:14.029 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:13.115+0000 7f8aa44b6640 1 -- 172.21.15.107:0/2208196904 wait complete. 2024-09-10T18:54:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:14 smithi107 ceph-mon[29404]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:14 smithi107 ceph-mon[40355]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:14.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:14 smithi162 ceph-mon[33228]: pgmap v9: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 17 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:14.593 INFO:teuthology.orchestra.run.smithi107.stdout:34359738417 2024-09-10T18:54:14.593 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-09-10T18:54:16.249 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/2373617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fe250 msgr2=0x7fbc14106890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/2373617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fe250 0x7fbc14106890 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fbc08004950 tx=0x7fbc08009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/2373617 shutdown_connections 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/2373617 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc14109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/2373617 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fe250 0x7fbc14106890 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/2373617 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/2373617 >> 172.21.15.107:0/2373617 conn(0x7fbc14071c00 msgr2=0x7fbc14072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/2373617 shutdown_connections 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.667+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/2373617 wait complete. 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 Processor -- start 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 -- start start 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:16.250 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fe250 0x7fbc140fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc140fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1411b430 con 0x7fbc140fd8b0 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1411b5a0 con 0x7fbc140fe250 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc1a6cf640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc1411b760 con 0x7fbc14106e60 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc18c6c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc140fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53288/0 (socket says 172.21.15.107:53288) 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc18c6c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc140fc160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:57466/0 (socket says 172.21.15.107:57466) 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 -- 172.21.15.107:0/657624113 learned_addr learned my addr 172.21.15.107:0/657624113 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc137fe640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fe250 0x7fbc140fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 -- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 msgr2=0x7fbc140fc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc140fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 -- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fe250 msgr2=0x7fbc140fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.251 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fe250 0x7fbc140fbc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc13fff640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc0c0098b0 con 0x7fbc140fd8b0 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc18c6c640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fbc14106e60 0x7fbc140fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.668+0000 7fbc137fe640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fbc140fe250 0x7fbc140fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc13fff640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fb6e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400b810 tx=0x7fbc040091d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc0400be60 con 0x7fbc140fd8b0 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fbc0400cb70 con 0x7fbc140fd8b0 2024-09-10T18:54:16.252 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc040159f0 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc08005040 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.669+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc1411bd80 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.670+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fbc140faaf0 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.670+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fbc04012070 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.671+0000 7fbc10ff9640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fbbfc06ef10 0x7fbbfc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.671+0000 7fbc137fe640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fbbfc06ef10 0x7fbbfc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.671+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fbc0408e0c0 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.671+0000 7fbc137fe640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fbbfc06ef10 0x7fbbfc0713c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc140fce10 tx=0x7fbc08035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.672+0000 7fbc10ff9640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] conn(0x7fbbfc078d90 0x7fbbfc07b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.672+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fbbfc07b890 con 0x7fbbfc078d90 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.672+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbc04090700 con 0x7fbc140fd8b0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.672+0000 7fbc18c6c640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] conn(0x7fbbfc078d90 0x7fbbfc07b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:16.253 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.673+0000 7fbc18c6c640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] conn(0x7fbbfc078d90 0x7fbbfc07b1c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.673+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== osd.7 v2:172.21.15.162:6824/3888567684 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fbbfc07b890 con 0x7fbbfc078d90 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.686+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 --> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fbc14065b20 con 0x7fbbfc078d90 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.686+0000 7fbc10ff9640 1 -- 172.21.15.107:0/657624113 <== osd.7 v2:172.21.15.162:6824/3888567684 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7fbc14065b20 con 0x7fbbfc078d90 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.686+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] conn(0x7fbbfc078d90 msgr2=0x7fbbfc07b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.162:6824/3888567684,v1:172.21.15.162:6825/3888567684] conn(0x7fbbfc078d90 0x7fbbfc07b1c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fbbfc06ef10 msgr2=0x7fbbfc0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fbbfc06ef10 0x7fbbfc0713c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fbc140fce10 tx=0x7fbc08035040 comp rx=0 tx=0).stop 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 msgr2=0x7fbc140fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 --2- 172.21.15.107:0/657624113 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fbc140fd8b0 0x7fbc140fb6e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fbc0400b810 tx=0x7fbc040091d0 comp rx=0 tx=0).stop 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc18c6c640 1 -- 172.21.15.107:0/657624113 reap_dead start 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 shutdown_connections 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 >> 172.21.15.107:0/657624113 conn(0x7fbc14071c00 msgr2=0x7fbc140fa160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 shutdown_connections 2024-09-10T18:54:16.254 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:15.687+0000 7fbc1a6cf640 1 -- 172.21.15.107:0/657624113 wait complete. 2024-09-10T18:54:16.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:16 smithi162 ceph-mon[33228]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:54:16.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:16.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:16 smithi107 ceph-mon[29404]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:54:16.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:16 smithi107 ceph-mon[40355]: pgmap v10: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 15 KiB/s rd, 0 B/s wr, 6 op/s 2024-09-10T18:54:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:17.286 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 -- 172.21.15.107:0/1327911693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 msgr2=0x7f8bf40fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/1327911693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf40fc3d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8be40048c0 tx=0x7f8be4009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 -- 172.21.15.107:0/1327911693 shutdown_connections 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/1327911693 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 0x7f8bf4108810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/1327911693 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf40fc3d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/1327911693 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 0x7f8bf4101e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 -- 172.21.15.107:0/1327911693 >> 172.21.15.107:0/1327911693 conn(0x7f8bf4074d40 msgr2=0x7f8bf40f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:17.287 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 -- 172.21.15.107:0/1327911693 shutdown_connections 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.136+0000 7f8bfbdad640 1 -- 172.21.15.107:0/1327911693 wait complete. 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 Processor -- start 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 -- start start 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf410e090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 0x7f8bf410e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 0x7f8bf410eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41192c0 con 0x7f8bf40fbf60 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf4119430 con 0x7f8bf4101a60 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfbdad640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf41195f0 con 0x7f8bf40fc910 2024-09-10T18:54:17.288 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf410e090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9348640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 0x7f8bf410e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf410e090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53316/0 (socket says 172.21.15.107:53316) 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 -- 172.21.15.107:0/2451120864 learned_addr learned my addr 172.21.15.107:0/2451120864 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfa34a640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 0x7f8bf410eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 -- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 msgr2=0x7f8bf410e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 0x7f8bf410e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 -- 172.21.15.107:0/2451120864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 msgr2=0x7f8bf410eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 0x7f8bf410eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9b49640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec0098b0 con 0x7f8bf40fbf60 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bfa34a640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8bf4101a60 0x7f8bf410eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.137+0000 7f8bf9348640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8bf40fc910 0x7f8bf410e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:17.289 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bf9b49640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf410e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0002ad0 tx=0x7f8bf0004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0007c30 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8be4005040 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f8bf000f040 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0009340 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.138+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf4119c60 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.139+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f8bf405de10 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.139+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f8bf00096f0 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bea7fc640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8be006ef10 0x7f8be00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f8bf008e8b0 con 0x7f8bf40fbf60 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bf9348640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8be006ef10 0x7f8be00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bea7fc640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] conn(0x7f8be0078d10 0x7f8be007b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f8be007b810 con 0x7f8be0078d10 2024-09-10T18:54:17.290 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8bf0093240 con 0x7f8bf40fbf60 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bfa34a640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] conn(0x7f8be0078d10 0x7f8be007b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bf9348640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8be006ef10 0x7f8be00713c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40fb640 tx=0x7f8be4035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.140+0000 7f8bfa34a640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] conn(0x7f8be0078d10 0x7f8be007b140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.141+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== osd.3 v2:172.21.15.107:6826/3842970087 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f8be007b810 con 0x7f8be0078d10 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.155+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 --> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f8bf40fd180 con 0x7f8be0078d10 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.155+0000 7f8bea7fc640 1 -- 172.21.15.107:0/2451120864 <== osd.3 v2:172.21.15.107:6826/3842970087 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f8bf40fd180 con 0x7f8be0078d10 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] conn(0x7f8be0078d10 msgr2=0x7f8be007b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6826/3842970087,v1:172.21.15.107:6827/3842970087] conn(0x7f8be0078d10 0x7f8be007b140 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8be006ef10 msgr2=0x7f8be00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f8be006ef10 0x7f8be00713c0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40fb640 tx=0x7f8be4035040 comp rx=0 tx=0).stop 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 msgr2=0x7f8bf410e090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.291 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 --2- 172.21.15.107:0/2451120864 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f8bf40fbf60 0x7f8bf410e090 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8bf0002ad0 tx=0x7f8bf0004600 comp rx=0 tx=0).stop 2024-09-10T18:54:17.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfa34a640 1 -- 172.21.15.107:0/2451120864 reap_dead start 2024-09-10T18:54:17.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 shutdown_connections 2024-09-10T18:54:17.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 >> 172.21.15.107:0/2451120864 conn(0x7f8bf4074d40 msgr2=0x7f8bf40720e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:17.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 shutdown_connections 2024-09-10T18:54:17.292 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.156+0000 7f8bfbdad640 1 -- 172.21.15.107:0/2451120864 wait complete. 2024-09-10T18:54:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- 172.21.15.107:0/1035051649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea580fc160 msgr2=0x7fea58108580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/1035051649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea580fc160 0x7fea58108580 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fea50009d00 tx=0x7fea50004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- 172.21.15.107:0/1035051649 shutdown_connections 2024-09-10T18:54:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/1035051649 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea580fc160 0x7fea58108580 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.864 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/1035051649 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fb7b0 0x7fea580fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/1035051649 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea581017d0 0x7fea58101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- 172.21.15.107:0/1035051649 >> 172.21.15.107:0/1035051649 conn(0x7fea58072420 msgr2=0x7fea580f99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- 172.21.15.107:0/1035051649 shutdown_connections 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- 172.21.15.107:0/1035051649 wait complete. 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 Processor -- start 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.856+0000 7fea5d3f2640 1 -- start start 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea580fb7b0 0x7fea58114bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 0x7fea581150f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.865 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 0x7fea5810ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810f390 con 0x7fea581017d0 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810f500 con 0x7fea580fb7b0 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea5810f6c0 con 0x7fea580fc160 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 0x7fea5810ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4ffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 0x7fea581150f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 0x7fea5810ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53324/0 (socket says 172.21.15.107:53324) 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4ffff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 0x7fea581150f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:57502/0 (socket says 172.21.15.107:57502) 2024-09-10T18:54:17.866 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 -- 172.21.15.107:0/376007684 learned_addr learned my addr 172.21.15.107:0/376007684 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea56ffd640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea580fb7b0 0x7fea58114bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 -- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 msgr2=0x7fea581150f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 0x7fea581150f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 -- 172.21.15.107:0/376007684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea580fb7b0 msgr2=0x7fea58114bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea580fb7b0 0x7fea58114bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea500098b0 con 0x7fea581017d0 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea56ffd640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fea580fb7b0 0x7fea58114bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:54:17.867 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4ffff640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fea580fc160 0x7fea581150f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea577fe640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 0x7fea5810ed40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea50004750 tx=0x7fea50004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea50038070 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fea50009510 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea5003c890 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea5810f940 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.857+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea5811bba0 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.858+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fea58066870 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fea50030070 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fea4406ef10 0x7fea440713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fea500af340 con 0x7fea581017d0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea56ffd640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fea4406ef10 0x7fea440713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] conn(0x7fea44078d90 0x7fea4407b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fea4407b890 con 0x7fea44078d90 2024-09-10T18:54:17.868 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.859+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7fea500b1980 con 0x7fea581017d0 2024-09-10T18:54:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.860+0000 7fea4ffff640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] conn(0x7fea44078d90 0x7fea4407b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.860+0000 7fea56ffd640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fea4406ef10 0x7fea440713c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea48004d60 tx=0x7fea4800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.860+0000 7fea4ffff640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] conn(0x7fea44078d90 0x7fea4407b1c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:17.869 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.860+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== osd.2 v2:172.21.15.107:6818/1138865377 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7fea4407b890 con 0x7fea44078d90 2024-09-10T18:54:17.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 --> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fea580fcef0 con 0x7fea44078d90 2024-09-10T18:54:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea4f7fe640 1 -- 172.21.15.107:0/376007684 <== osd.2 v2:172.21.15.107:6818/1138865377 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fea580fcef0 con 0x7fea44078d90 2024-09-10T18:54:17.876 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] conn(0x7fea44078d90 msgr2=0x7fea4407b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6818/1138865377,v1:172.21.15.107:6819/1138865377] conn(0x7fea44078d90 0x7fea4407b1c0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fea4406ef10 msgr2=0x7fea440713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fea4406ef10 0x7fea440713c0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fea48004d60 tx=0x7fea4800b040 comp rx=0 tx=0).stop 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 msgr2=0x7fea5810ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 --2- 172.21.15.107:0/376007684 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fea581017d0 0x7fea5810ed40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fea50004750 tx=0x7fea50004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea577fe640 1 -- 172.21.15.107:0/376007684 reap_dead start 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 shutdown_connections 2024-09-10T18:54:17.877 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.873+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 >> 172.21.15.107:0/376007684 conn(0x7fea58072420 msgr2=0x7fea58065ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:17.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.874+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 shutdown_connections 2024-09-10T18:54:17.878 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:17.874+0000 7fea5d3f2640 1 -- 172.21.15.107:0/376007684 wait complete. 2024-09-10T18:54:18.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:18 smithi107 ceph-mon[29404]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:18.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:18 smithi107 ceph-mon[40355]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:18 smithi162 ceph-mon[33228]: pgmap v11: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:19.188 INFO:teuthology.orchestra.run.smithi107.stdout:180388626450 2024-09-10T18:54:19.188 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-09-10T18:54:19.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 -- 172.21.15.107:0/2392883768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c0fb810 msgr2=0x7f6b0c0fbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 --2- 172.21.15.107:0/2392883768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c0fb810 0x7f6b0c0fbc80 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6af4004980 tx=0x7f6af4009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 -- 172.21.15.107:0/2392883768 shutdown_connections 2024-09-10T18:54:19.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 --2- 172.21.15.107:0/2392883768 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fc1c0 0x7f6b0c1085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 --2- 172.21.15.107:0/2392883768 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c0fb810 0x7f6b0c0fbc80 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 --2- 172.21.15.107:0/2392883768 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b0c101830 0x7f6b0c101c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 -- 172.21.15.107:0/2392883768 >> 172.21.15.107:0/2392883768 conn(0x7f6b0c0713f0 msgr2=0x7f6b0c0f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 -- 172.21.15.107:0/2392883768 shutdown_connections 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 -- 172.21.15.107:0/2392883768 wait complete. 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.801+0000 7f6b120a2640 1 Processor -- start 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 -- start start 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 0x7f6b0c111bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b0c0fc1c0 0x7f6b0c112110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c101830 0x7f6b0c10bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c10c3b0 con 0x7f6b0c0fb810 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 0x7f6b0c111bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c101830 0x7f6b0c10bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0b7fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 0x7f6b0c111bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53344/0 (socket says 172.21.15.107:53344) 2024-09-10T18:54:19.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0b7fe640 1 -- 172.21.15.107:0/1991748868 learned_addr learned my addr 172.21.15.107:0/1991748868 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c10c520 con 0x7f6b0c0fc1c0 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b0c10c6e0 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0affd640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b0c0fc1c0 0x7f6b0c112110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 -- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b0c0fc1c0 msgr2=0x7f6b0c112110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6b0c0fc1c0 0x7f6b0c112110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 -- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 msgr2=0x7f6b0c111bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 0x7f6b0c111bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0bfff640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b040098b0 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.802+0000 7f6b0b7fe640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6b0c0fb810 0x7f6b0c111bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b0bfff640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c101830 0x7f6b0c10bd60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b04001da0 tx=0x7f6b04004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b0400fdd0 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f6b04007cd0 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af4005040 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04009610 con 0x7f6b0c101830 2024-09-10T18:54:19.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.803+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b0c118a20 con 0x7f6b0c101830 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.804+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6b0c05dd90 con 0x7f6b0c101830 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.804+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f6b04007890 con 0x7f6b0c101830 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b03fff640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6af806ef10 0x7f6af80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b0b7fe640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6af806ef10 0x7f6af80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f6b0408e890 con 0x7f6b0c101830 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b03fff640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] conn(0x7f6af8078d90 0x7f6af807b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6af807b890 con 0x7f6af8078d90 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6b04093240 con 0x7f6b0c101830 2024-09-10T18:54:19.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b0b7fe640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6af806ef10 0x7f6af80713c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6afc004fd0 tx=0x7f6afc00b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.805+0000 7f6b0affd640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] conn(0x7f6af8078d90 0x7f6af807b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.806+0000 7f6b0affd640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] conn(0x7f6af8078d90 0x7f6af807b1c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.806+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== osd.5 v2:172.21.15.162:6808/688086025 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f6af807b890 con 0x7f6af8078d90 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.819+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 --> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6b0c0fcf50 con 0x7f6af8078d90 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.819+0000 7f6b03fff640 1 -- 172.21.15.107:0/1991748868 <== osd.5 v2:172.21.15.162:6808/688086025 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+12 (crc 0 0 0) 0x7f6b0c0fcf50 con 0x7f6af8078d90 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] conn(0x7f6af8078d90 msgr2=0x7f6af807b1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.162:6808/688086025,v1:172.21.15.162:6809/688086025] conn(0x7f6af8078d90 0x7f6af807b1c0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6af806ef10 msgr2=0x7f6af80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6af806ef10 0x7f6af80713c0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6afc004fd0 tx=0x7f6afc00b040 comp rx=0 tx=0).stop 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c101830 msgr2=0x7f6b0c10bd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 --2- 172.21.15.107:0/1991748868 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6b0c101830 0x7f6b0c10bd60 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6b04001da0 tx=0x7f6b04004c50 comp rx=0 tx=0).stop 2024-09-10T18:54:19.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b0bfff640 1 -- 172.21.15.107:0/1991748868 reap_dead start 2024-09-10T18:54:19.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 shutdown_connections 2024-09-10T18:54:19.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 >> 172.21.15.107:0/1991748868 conn(0x7f6b0c0713f0 msgr2=0x7f6b0c0728b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:19.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 shutdown_connections 2024-09-10T18:54:19.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:19.820+0000 7f6b120a2640 1 -- 172.21.15.107:0/1991748868 wait complete. 2024-09-10T18:54:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:20 smithi162 ceph-mon[33228]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:20.436 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:20 smithi107 ceph-mon[29404]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:20.436 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:20 smithi107 ceph-mon[40355]: pgmap v12: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 16 KiB/s rd, 0 B/s wr, 7 op/s 2024-09-10T18:54:20.458 INFO:teuthology.orchestra.run.smithi107.stdout:206158430223 2024-09-10T18:54:20.459 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-09-10T18:54:20.870 INFO:teuthology.orchestra.run.smithi107.stdout:128849018909 2024-09-10T18:54:20.870 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-09-10T18:54:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:54:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:21.511 INFO:teuthology.orchestra.run.smithi107.stdout:77309411367 2024-09-10T18:54:21.511 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-09-10T18:54:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- 172.21.15.107:0/3539739185 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fd8b0 msgr2=0x7f7f0c0fdc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 --2- 172.21.15.107:0/3539739185 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fd8b0 0x7f7f0c0fdc80 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f7f08004890 tx=0x7f7f08009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- 172.21.15.107:0/3539739185 shutdown_connections 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 --2- 172.21.15.107:0/3539739185 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 0x7f7f0c109240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 --2- 172.21.15.107:0/3539739185 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fe250 0x7f7f0c106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 --2- 172.21.15.107:0/3539739185 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fd8b0 0x7f7f0c0fdc80 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- 172.21.15.107:0/3539739185 >> 172.21.15.107:0/3539739185 conn(0x7f7f0c071c00 msgr2=0x7f7f0c072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- 172.21.15.107:0/3539739185 shutdown_connections 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- 172.21.15.107:0/3539739185 wait complete. 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 Processor -- start 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.683+0000 7f7f137d4640 1 -- start start 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fd8b0 0x7f7f0c112d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 0x7f7f0c113260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:21.688 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 0x7f7f0c10ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c10d500 con 0x7f7f0c0fd8b0 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c10d670 con 0x7f7f0c106e60 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f137d4640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f0c10d830 con 0x7f7f0c0fe250 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 0x7f7f0c113260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 0x7f7f0c113260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:57540/0 (socket says 172.21.15.107:57540) 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 -- 172.21.15.107:0/1696157353 learned_addr learned my addr 172.21.15.107:0/1696157353 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f11570640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fd8b0 0x7f7f0c112d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f11d71640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 0x7f7f0c10ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 -- 172.21.15.107:0/1696157353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 msgr2=0x7f7f0c10ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 0x7f7f0c10ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 -- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fd8b0 msgr2=0x7f7f0c112d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.689 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fd8b0 0x7f7f0c112d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7efc005040 con 0x7f7f0c0fe250 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f11d71640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7f0c106e60 0x7f7f0c10ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f11570640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7f0c0fd8b0 0x7f7f0c112d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f10d6f640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 0x7f7f0c113260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009550 tx=0x7f7efc009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.684+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc00e930 con 0x7f7f0c0fe250 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.685+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f08005040 con 0x7f7f0c0fe250 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.685+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f7efc00f040 con 0x7f7f0c0fe250 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.685+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7efc013b10 con 0x7f7f0c0fe250 2024-09-10T18:54:21.690 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.685+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f0c119b70 con 0x7f7f0c0fe250 2024-09-10T18:54:21.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.686+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7f0c0faa90 con 0x7f7f0c0fe250 2024-09-10T18:54:21.691 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f7efc00ea90 con 0x7f7f0c0fe250 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7ef806ef10 0x7f7ef80713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f7efc08f000 con 0x7f7f0c0fe250 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f11570640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7ef806ef10 0x7f7ef80713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] conn(0x7f7ef8078d10 0x7f7ef807b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7ef807b810 con 0x7f7ef8078d10 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.687+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_get_version_reply(handle=1 version=61) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7efc08f3d0 con 0x7f7f0c0fe250 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.688+0000 7f7f11d71640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] conn(0x7f7ef8078d10 0x7f7ef807b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.688+0000 7f7f11570640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7ef806ef10 0x7f7ef80713c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009040 tx=0x7f7f08035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.688+0000 7f7f11d71640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] conn(0x7f7ef8078d10 0x7f7ef807b140 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:21.692 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.688+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== osd.1 v2:172.21.15.107:6810/462447353 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+24147 (crc 0 0 0) 0x7f7ef807b810 con 0x7f7ef8078d10 2024-09-10T18:54:21.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.702+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 --> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f7f0c0fae40 con 0x7f7ef8078d10 2024-09-10T18:54:21.704 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.702+0000 7f7f01ffb640 1 -- 172.21.15.107:0/1696157353 <== osd.1 v2:172.21.15.107:6810/462447353 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f7f0c0fae40 con 0x7f7ef8078d10 2024-09-10T18:54:21.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.702+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] conn(0x7f7ef8078d10 msgr2=0x7f7ef807b140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.702+0000 7f7f137d4640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6810/462447353,v1:172.21.15.107:6811/462447353] conn(0x7f7ef8078d10 0x7f7ef807b140 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.705 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7ef806ef10 msgr2=0x7f7ef80713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7ef806ef10 0x7f7ef80713c0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7f08009040 tx=0x7f7f08035040 comp rx=0 tx=0).stop 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 msgr2=0x7f7f0c113260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 --2- 172.21.15.107:0/1696157353 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7f0c0fe250 0x7f7f0c113260 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7efc009550 tx=0x7f7efc009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f11d71640 1 -- 172.21.15.107:0/1696157353 reap_dead start 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 shutdown_connections 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 >> 172.21.15.107:0/1696157353 conn(0x7f7f0c071c00 msgr2=0x7f7f0c0fa100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 shutdown_connections 2024-09-10T18:54:21.706 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:21.703+0000 7f7f137d4640 1 -- 172.21.15.107:0/1696157353 wait complete. 2024-09-10T18:54:22.402 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:22 smithi107 ceph-mon[29404]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:22.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:22 smithi107 ceph-mon[40355]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:22.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:22 smithi162 ceph-mon[33228]: pgmap v13: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:22.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:22.426 INFO:teuthology.orchestra.run.smithi107.stdout:55834574892 2024-09-10T18:54:22.426 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-09-10T18:54:22.869 INFO:teuthology.orchestra.run.smithi107.stdout:154618822682 2024-09-10T18:54:22.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-09-10T18:54:23.314 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:23.349 INFO:teuthology.orchestra.run.smithi107.stdout:107374182434 2024-09-10T18:54:23.349 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-09-10T18:54:23.373 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:23.374 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:23.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:24.102 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:24.103 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:24.104 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:24.372 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:24 smithi162 ceph-mon[33228]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:24 smithi107 ceph-mon[29404]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:24.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:24 smithi107 ceph-mon[40355]: pgmap v14: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:25.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:25 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:25.176 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:25 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:25.177 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:25 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:25.177 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:25 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:25.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:25 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:25.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:25 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:26.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:26 smithi107 ceph-mon[29404]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:26 smithi107 ceph-mon[40355]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:26 smithi162 ceph-mon[33228]: pgmap v15: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:26.924 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 -- 172.21.15.107:0/3733738119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b881017e0 msgr2=0x7f2b88101bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/3733738119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b881017e0 0x7f2b88101bb0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f2b78004890 tx=0x7f2b78009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 -- 172.21.15.107:0/3733738119 shutdown_connections 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/3733738119 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fc2e0 0x7f2b88108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/3733738119 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b880fb950 0x7f2b880fbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/3733738119 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b881017e0 0x7f2b88101bb0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 -- 172.21.15.107:0/3733738119 >> 172.21.15.107:0/3733738119 conn(0x7f2b88072540 msgr2=0x7f2b88072940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 -- 172.21.15.107:0/3733738119 shutdown_connections 2024-09-10T18:54:27.057 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.699+0000 7f2b8cca0640 1 -- 172.21.15.107:0/3733738119 wait complete. 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 Processor -- start 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 -- start start 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b880fc2e0 0x7f2b8810e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 0x7f2b8810e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88118fd0 con 0x7f2b880fb950 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88119140 con 0x7f2b881017e0 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8cca0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b88119300 con 0x7f2b880fc2e0 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:49984/0 (socket says 172.21.15.107:49984) 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 -- 172.21.15.107:0/2334773797 learned_addr learned my addr 172.21.15.107:0/2334773797 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b85d9b640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b880fc2e0 0x7f2b8810e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:27.058 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b86d9d640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 0x7f2b8810e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 -- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b880fc2e0 msgr2=0x7f2b8810e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b880fc2e0 0x7f2b8810e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 -- 172.21.15.107:0/2334773797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 msgr2=0x7f2b8810e7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.700+0000 7f2b8659c640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 0x7f2b8810e7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b8659c640 1 -- 172.21.15.107:0/2334773797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b800098b0 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b86d9d640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 0x7f2b8810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b8659c640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b7802a4a0 tx=0x7f2b78009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b78000c90 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b78005040 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f2b78033860 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b7803b5f0 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.701+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b88119990 con 0x7f2b880fb950 2024-09-10T18:54:27.059 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.702+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b880fcf00 con 0x7f2b880fb950 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.702+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f2b78030020 con 0x7f2b880fb950 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.703+0000 7f2b76ffd640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 0x7f2b700713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.703+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f2b780ae730 con 0x7f2b880fb950 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.703+0000 7f2b85d9b640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 0x7f2b700713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.703+0000 7f2b85d9b640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 0x7f2b700713c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2b88066770 tx=0x7f2b7c009040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:27.060 INFO:teuthology.orchestra.run.smithi107.stdout:34359738419 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.707+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f2b7807b1e0 con 0x7f2b880fb950 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.871+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2b8810aa50 con 0x7f2b880fb950 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.871+0000 7f2b76ffd640 1 -- 172.21.15.107:0/2334773797 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2b7807ab80 con 0x7f2b880fb950 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 msgr2=0x7f2b700713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 0x7f2b700713c0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2b88066770 tx=0x7f2b7c009040 comp rx=0 tx=0).stop 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 msgr2=0x7f2b8810dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2b7802a4a0 tx=0x7f2b78009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 shutdown_connections 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f2b7006ef10 0x7f2b700713c0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f2b881017e0 0x7f2b8810e7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.061 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f2b880fc2e0 0x7f2b8810e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 --2- 172.21.15.107:0/2334773797 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f2b880fb950 0x7f2b8810dd50 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:27.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 >> 172.21.15.107:0/2334773797 conn(0x7f2b88072540 msgr2=0x7f2b880fa620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:27.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 shutdown_connections 2024-09-10T18:54:27.062 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:26.874+0000 7f2b8cca0640 1 -- 172.21.15.107:0/2334773797 wait complete. 2024-09-10T18:54:27.178 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2334773797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:54:27.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2334773797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:54:27.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2334773797' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-09-10T18:54:28.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:28 smithi162 ceph-mon[33228]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:28 smithi107 ceph-mon[29404]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:28.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:28 smithi107 ceph-mon[40355]: pgmap v16: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:29.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:54:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:29 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 3759M 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: Updating smithi107:/etc/ceph/ceph.conf 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: Updating smithi162:/etc/ceph/ceph.conf 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.conf 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:29.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:29 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[29404]: Deploying daemon alertmanager.a on smithi107 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:30 smithi107 ceph-mon[40355]: Deploying daemon alertmanager.a on smithi107 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: Updating smithi107:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: Updating smithi162:/etc/ceph/ceph.client.admin.keyring 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: Updating smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: Updating smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/config/ceph.client.admin.keyring 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: pgmap v17: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-mon[33228]: Deploying daemon alertmanager.a on smithi107 2024-09-10T18:54:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:54:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:31.414 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:31.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:32 smithi162 ceph-mon[33228]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:32.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:32.425 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738417 got 34359738419 for osd.0 2024-09-10T18:54:32.425 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:32 smithi107 ceph-mon[29404]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:32 smithi107 ceph-mon[40355]: pgmap v18: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:32.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 -- 172.21.15.107:0/675820162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 msgr2=0x7fb924102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 --2- 172.21.15.107:0/675820162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb924102cf0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9140048c0 tx=0x7fb914009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 -- 172.21.15.107:0/675820162 shutdown_connections 2024-09-10T18:54:32.728 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 --2- 172.21.15.107:0/675820162 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fb730 0x7fb9241096c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 --2- 172.21.15.107:0/675820162 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fada0 0x7fb9240fb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 --2- 172.21.15.107:0/675820162 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb924102cf0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 -- 172.21.15.107:0/675820162 >> 172.21.15.107:0/675820162 conn(0x7fb924074d70 msgr2=0x7fb9240f9130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 -- 172.21.15.107:0/675820162 shutdown_connections 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.721+0000 7fb92a751640 1 -- 172.21.15.107:0/675820162 wait complete. 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 Processor -- start 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 -- start start 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fada0 0x7fb924112cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb92410ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92410d4a0 con 0x7fb9240fb730 2024-09-10T18:54:32.729 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb92410d610 con 0x7fb9240fada0 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb92a751640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb92410d7d0 con 0x7fb924102920 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb9237fe640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50020/0 (socket says 172.21.15.107:50020) 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb9237fe640 1 -- 172.21.15.107:0/3597969367 learned_addr learned my addr 172.21.15.107:0/3597969367 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb928cee640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb92410ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.722+0000 7fb923fff640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fada0 0x7fb924112cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 -- 172.21.15.107:0/3597969367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fada0 msgr2=0x7fb924112cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fada0 0x7fb924112cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 -- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 msgr2=0x7fb924113200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 -- 172.21.15.107:0/3597969367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb90c005040 con 0x7fb924102920 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb9237fe640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb928cee640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb92410ce50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00db80 tx=0x7fb91c004db0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:32.730 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c017070 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb914005040 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fb91c00f040 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb91c0036a0 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.723+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb924119b10 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.724+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9240fe0c0 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.725+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fb91c003ad0 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.725+0000 7fb920ff9640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 0x7fb9100713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.725+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fb91c08e8f0 con 0x7fb924102920 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.725+0000 7fb923fff640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 0x7fb9100713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.731 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.726+0000 7fb923fff640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 0x7fb9100713c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb9140047e0 tx=0x7fb914035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:32.732 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.730+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb91c05b320 con 0x7fb924102920 2024-09-10T18:54:32.824 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 -- 172.21.15.107:0/2937040671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac0fc300 msgr2=0x7ff3ac108560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2937040671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac0fc300 0x7ff3ac108560 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009d00 tx=0x7ff3a4004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 -- 172.21.15.107:0/2937040671 shutdown_connections 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2937040671 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac0fc300 0x7ff3ac108560 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2937040671 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fbdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2937040671 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac1017b0 0x7ff3ac101b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 -- 172.21.15.107:0/2937040671 >> 172.21.15.107:0/2937040671 conn(0x7ff3ac0724d0 msgr2=0x7ff3ac0728d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 -- 172.21.15.107:0/2937040671 shutdown_connections 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.631+0000 7ff3b2450640 1 -- 172.21.15.107:0/2937040671 wait complete. 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 Processor -- start 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 -- start start 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 0x7ff3ac0fd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac11b060 con 0x7ff3ac1017b0 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac11b1d0 con 0x7ff3ac0fb950 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b2450640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3ac11b390 con 0x7ff3ac0fc300 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 0x7ff3ac0fd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50004/0 (socket says 172.21.15.107:50004) 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 -- 172.21.15.107:0/2671589098 learned_addr learned my addr 172.21.15.107:0/2671589098 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3a3fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 0x7ff3ac0fd8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:50324/0 (socket says 172.21.15.107:50324) 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3abfff640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 -- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 msgr2=0x7ff3ac0fd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 0x7ff3ac0fd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 -- 172.21.15.107:0/2671589098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 msgr2=0x7ff3ac0fd3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fd3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3b09ed640 1 -- 172.21.15.107:0/2671589098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff39c005040 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.632+0000 7ff3abfff640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3b09ed640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009cd0 tx=0x7ff3a4004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a4038070 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3a40098b0 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7ff3a4009510 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a403d960 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.633+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3ac11b940 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.634+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3ac0fced0 con 0x7ff3ac1017b0 2024-09-10T18:54:32.827 INFO:teuthology.orchestra.run.smithi107.stdout:180388626454 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.634+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7ff3a4030070 con 0x7ff3ac1017b0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.635+0000 7ff3a97fa640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 0x7ff3980713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.635+0000 7ff3abfff640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 0x7ff3980713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.635+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7ff3a40af820 con 0x7ff3ac1017b0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.636+0000 7ff3abfff640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 0x7ff3980713c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff39c007fd0 tx=0x7ff39c009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.639+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7ff3a407c1a0 con 0x7ff3ac1017b0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.805+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff3ac10a990 con 0x7ff3ac1017b0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.806+0000 7ff3a97fa640 1 -- 172.21.15.107:0/2671589098 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7ff3a407bb40 con 0x7ff3ac1017b0 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 msgr2=0x7ff3980713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 0x7ff3980713c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff39c007fd0 tx=0x7ff39c009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 msgr2=0x7ff3ac0fde20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff3a4009cd0 tx=0x7ff3a4004d10 comp rx=0 tx=0).stop 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 shutdown_connections 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7ff39806ef10 0x7ff3980713c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7ff3ac1017b0 0x7ff3ac0fde20 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7ff3ac0fc300 0x7ff3ac0fd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 --2- 172.21.15.107:0/2671589098 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7ff3ac0fb950 0x7ff3ac0fd3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 >> 172.21.15.107:0/2671589098 conn(0x7ff3ac0724d0 msgr2=0x7ff3ac0fa510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 shutdown_connections 2024-09-10T18:54:32.829 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.808+0000 7ff3b2450640 1 -- 172.21.15.107:0/2671589098 wait complete. 2024-09-10T18:54:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.893+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fb92410bbd0 con 0x7fb924102920 2024-09-10T18:54:32.895 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.893+0000 7fb920ff9640 1 -- 172.21.15.107:0/3597969367 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb91c05acc0 con 0x7fb924102920 2024-09-10T18:54:32.895 INFO:teuthology.orchestra.run.smithi107.stdout:77309411369 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 msgr2=0x7fb9100713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 0x7fb9100713c0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fb9140047e0 tx=0x7fb914035040 comp rx=0 tx=0).stop 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 msgr2=0x7fb92410ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb92410ce50 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fb91c00db80 tx=0x7fb91c004db0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 shutdown_connections 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb91006ef10 0x7fb9100713c0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.897 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb924102920 0x7fb92410ce50 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb9240fb730 0x7fb924113200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 --2- 172.21.15.107:0/3597969367 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb9240fada0 0x7fb924112cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.895+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 >> 172.21.15.107:0/3597969367 conn(0x7fb924074d70 msgr2=0x7fb9240720f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.896+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 shutdown_connections 2024-09-10T18:54:32.898 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:32.896+0000 7fb92a751640 1 -- 172.21.15.107:0/3597969367 wait complete. 2024-09-10T18:54:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2671589098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:54:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3597969367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:54:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2671589098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:54:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3597969367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:54:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2671589098' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-09-10T18:54:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3597969367' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.627+0000 7f9a14c95640 1 -- 172.21.15.107:0/2410198823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 msgr2=0x7f9a100fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.627+0000 7f9a14c95640 1 --2- 172.21.15.107:0/2410198823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a100fbc70 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f9a00004980 tx=0x7f9a00009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- 172.21.15.107:0/2410198823 shutdown_connections 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- 172.21.15.107:0/2410198823 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a100fc1b0 0x7f9a10108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- 172.21.15.107:0/2410198823 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a100fbc70 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- 172.21.15.107:0/2410198823 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a101017d0 0x7f9a10101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- 172.21.15.107:0/2410198823 >> 172.21.15.107:0/2410198823 conn(0x7f9a10071c00 msgr2=0x7f9a10072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:33.836 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- 172.21.15.107:0/2410198823 shutdown_connections 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- 172.21.15.107:0/2410198823 wait complete. 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 Processor -- start 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- start start 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a100fc1b0 0x7f9a1010e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 0x7f9a1010e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a10119030 con 0x7f9a100fb800 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a101191a0 con 0x7f9a101017d0 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a14c95640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a10119360 con 0x7f9a100fc1b0 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.628+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50040/0 (socket says 172.21.15.107:50040) 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 -- 172.21.15.107:0/3593541008 learned_addr learned my addr 172.21.15.107:0/3593541008 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:33.837 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0dd9b640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a100fc1b0 0x7f9a1010e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:33.838 INFO:teuthology.orchestra.run.smithi107.stdout:107374182437 2024-09-10T18:54:33.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0ed9d640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 0x7f9a1010e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:33.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 -- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a100fc1b0 msgr2=0x7f9a1010e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:33.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a100fc1b0 0x7f9a1010e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.838 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 -- 172.21.15.107:0/3593541008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 msgr2=0x7f9a1010e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 0x7f9a1010e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 -- 172.21.15.107:0/3593541008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99f8005040 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0ed9d640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 0x7f9a1010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a0e59c640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009780 tx=0x7f99f8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f80099c0 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f99f800f040 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a00005040 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99f8013bf0 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.629+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a101199a0 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.630+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a100fcef0 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.630+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f99f800ebb0 con 0x7f9a100fb800 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.631+0000 7f9a06ffd640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 0x7f99fc0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:33.839 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.631+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f99f808e960 con 0x7f9a100fb800 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.631+0000 7f9a0dd9b640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 0x7f99fc0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.631+0000 7f9a0dd9b640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 0x7f99fc0713c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9a10065aa0 tx=0x7f9a00035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.635+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f99f805b2e0 con 0x7f9a100fb800 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.799+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f9a10065850 con 0x7f9a100fb800 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.799+0000 7f9a06ffd640 1 -- 172.21.15.107:0/3593541008 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f99f805ac80 con 0x7f9a100fb800 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 msgr2=0x7f99fc0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 0x7f99fc0713c0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9a10065aa0 tx=0x7f9a00035040 comp rx=0 tx=0).stop 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 msgr2=0x7f9a1010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f99f8009780 tx=0x7f99f8009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 shutdown_connections 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f99fc06ef10 0x7f99fc0713c0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.840 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9a101017d0 0x7f9a1010e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9a100fc1b0 0x7f9a1010e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 --2- 172.21.15.107:0/3593541008 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9a100fb800 0x7f9a1010dd80 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:33.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.801+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 >> 172.21.15.107:0/3593541008 conn(0x7f9a10071c00 msgr2=0x7f9a100fa2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:33.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.802+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 shutdown_connections 2024-09-10T18:54:33.841 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:33.802+0000 7f9a14c95640 1 -- 172.21.15.107:0/3593541008 wait complete. 2024-09-10T18:54:34.059 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:34.457 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:34 smithi107 ceph-mon[29404]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:34.457 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3593541008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:54:34.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:34 smithi107 ceph-mon[40355]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:34.457 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3593541008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:54:34.480 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182434 got 107374182437 for osd.3 2024-09-10T18:54:34.480 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:34.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:34 smithi162 ceph-mon[33228]: pgmap v19: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3593541008' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- 172.21.15.107:0/3575354799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 msgr2=0x7f37280fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 --2- 172.21.15.107:0/3575354799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f37280fbc70 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f371c0048c0 tx=0x7f371c009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- 172.21.15.107:0/3575354799 shutdown_connections 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 --2- 172.21.15.107:0/3575354799 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37280fc1b0 0x7f3728108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 --2- 172.21.15.107:0/3575354799 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f37280fbc70 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 --2- 172.21.15.107:0/3575354799 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37281017d0 0x7f3728101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- 172.21.15.107:0/3575354799 >> 172.21.15.107:0/3575354799 conn(0x7f3728071c00 msgr2=0x7f3728072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- 172.21.15.107:0/3575354799 shutdown_connections 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- 172.21.15.107:0/3575354799 wait complete. 2024-09-10T18:54:34.684 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 Processor -- start 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.678+0000 7f372ea04640 1 -- start start 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37280fc1b0 0x7f3728112060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 0x7f372810bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372810c300 con 0x7f37281017d0 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f372810c470 con 0x7f37280fc1b0 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f372810c630 con 0x7f37280fb800 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372cfa1640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 0x7f372810bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:54516/0 (socket says 172.21.15.107:54516) 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 -- 172.21.15.107:0/3373086950 learned_addr learned my addr 172.21.15.107:0/3373086950 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f37277fe640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37280fc1b0 0x7f3728112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:34.685 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 -- 172.21.15.107:0/3373086950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37280fc1b0 msgr2=0x7f3728112060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37280fc1b0 0x7f3728112060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 -- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 msgr2=0x7f372810bcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 0x7f372810bcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 -- 172.21.15.107:0/3373086950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37200098b0 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372cfa1640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 0x7f372810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3727fff640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3718002ad0 tx=0x7f3718004600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718007c30 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f371800f040 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3718009340 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f371c005040 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.679+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3728118970 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.681+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37280fcef0 con 0x7f37280fb800 2024-09-10T18:54:34.686 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.681+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f3718009770 con 0x7f37280fb800 2024-09-10T18:54:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.681+0000 7f3724ff9640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 0x7f37100713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.681+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f371808e660 con 0x7f37280fb800 2024-09-10T18:54:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.681+0000 7f37277fe640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 0x7f37100713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.682+0000 7f37277fe640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 0x7f37100713c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f371c0047e0 tx=0x7f371c035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:34.687 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.686+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f371805b1d0 con 0x7f37280fb800 2024-09-10T18:54:34.847 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.846+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f372810aa40 con 0x7f37280fb800 2024-09-10T18:54:34.848 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.846+0000 7f3724ff9640 1 -- 172.21.15.107:0/3373086950 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f371805ab70 con 0x7f37280fb800 2024-09-10T18:54:34.848 INFO:teuthology.orchestra.run.smithi107.stdout:55834574895 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 msgr2=0x7f37100713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 0x7f37100713c0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f371c0047e0 tx=0x7f371c035040 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 msgr2=0x7f3728111b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f3718002ad0 tx=0x7f3718004600 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 shutdown_connections 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f371006ef10 0x7f37100713c0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f37281017d0 0x7f372810bcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f37280fc1b0 0x7f3728112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 --2- 172.21.15.107:0/3373086950 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f37280fb800 0x7f3728111b20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 >> 172.21.15.107:0/3373086950 conn(0x7f3728071c00 msgr2=0x7f37280fa2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:34.850 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 shutdown_connections 2024-09-10T18:54:34.851 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:34.848+0000 7f372ea04640 1 -- 172.21.15.107:0/3373086950 wait complete. 2024-09-10T18:54:35.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3373086950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:54:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3373086950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:54:35.565 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574892 got 55834574895 for osd.1 2024-09-10T18:54:35.565 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:35.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3373086950' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-09-10T18:54:36.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411367 got 77309411369 for osd.2 2024-09-10T18:54:36.147 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:36.456 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:36 smithi107 ceph-mon[29404]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:36.456 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:36 smithi107 ceph-mon[40355]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:36.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:36 smithi162 ceph-mon[33228]: pgmap v20: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 -- 172.21.15.107:0/685295447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 msgr2=0x7f80e40fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 --2- 172.21.15.107:0/685295447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e40fbc70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f80cc004980 tx=0x7f80cc009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 -- 172.21.15.107:0/685295447 shutdown_connections 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 --2- 172.21.15.107:0/685295447 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e40fc1b0 0x7f80e4108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 --2- 172.21.15.107:0/685295447 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e40fbc70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 --2- 172.21.15.107:0/685295447 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e41017d0 0x7f80e4101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 -- 172.21.15.107:0/685295447 >> 172.21.15.107:0/685295447 conn(0x7f80e4071c00 msgr2=0x7f80e4072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 -- 172.21.15.107:0/685295447 shutdown_connections 2024-09-10T18:54:37.044 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.962+0000 7f80ea18d640 1 -- 172.21.15.107:0/685295447 wait complete. 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 Processor -- start 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 -- start start 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e41017d0 0x7f80e410cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d440 con 0x7f80e40fc1b0 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d5b0 con 0x7f80e41017d0 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80ea18d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80e410d770 con 0x7f80e40fb800 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80e37fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:37.045 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.963+0000 7f80dbfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37570/0 (socket says 172.21.15.107:37570) 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e37fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:54532/0 (socket says 172.21.15.107:54532) 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 -- 172.21.15.107:0/3519779590 learned_addr learned my addr 172.21.15.107:0/3519779590 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e3fff640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e41017d0 0x7f80e410cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 -- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 msgr2=0x7f80e4112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 -- 172.21.15.107:0/3519779590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e41017d0 msgr2=0x7f80e410cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e41017d0 0x7f80e410cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 -- 172.21.15.107:0/3519779590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80d4005040 con 0x7f80e40fc1b0 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e37fe640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80dbfff640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f80cc02d040 tx=0x7f80cc035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc02aca0 con 0x7f80e40fc1b0 2024-09-10T18:54:37.046 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f80cc02ae00 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f80cc005040 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80cc03c410 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.964+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80e4119ab0 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.965+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80e40fcef0 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.966+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f80cc030020 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.966+0000 7f80e0ff9640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 0x7f80d00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.966+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f80cc0ae5d0 con 0x7f80e40fc1b0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.966+0000 7f80e37fe640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 0x7f80d00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.967+0000 7f80e37fe640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 0x7f80d00713c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f80d4004e50 tx=0x7f80d400b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:37.047 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:36.970+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f80cc07b220 con 0x7f80e40fc1b0 2024-09-10T18:54:37.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.132+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f80e410e300 con 0x7f80e40fc1b0 2024-09-10T18:54:37.134 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.133+0000 7f80e0ff9640 1 -- 172.21.15.107:0/3519779590 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f80cc07abc0 con 0x7f80e40fc1b0 2024-09-10T18:54:37.134 INFO:teuthology.orchestra.run.smithi107.stdout:206158430227 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 msgr2=0x7f80d00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 0x7f80d00713c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f80d4004e50 tx=0x7f80d400b040 comp rx=0 tx=0).stop 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 msgr2=0x7f80e41131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f80cc02d040 tx=0x7f80cc035040 comp rx=0 tx=0).stop 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 shutdown_connections 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f80d006ef10 0x7f80d00713c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80e41017d0 0x7f80e410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80e40fc1b0 0x7f80e41131a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.137 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 --2- 172.21.15.107:0/3519779590 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80e40fb800 0x7f80e4112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:37.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 >> 172.21.15.107:0/3519779590 conn(0x7f80e4071c00 msgr2=0x7f80e40fa380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:37.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 shutdown_connections 2024-09-10T18:54:37.138 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:37.135+0000 7f80ea18d640 1 -- 172.21.15.107:0/3519779590 wait complete. 2024-09-10T18:54:37.432 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3519779590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:54:37.433 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3519779590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:54:37.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3519779590' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-09-10T18:54:38.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:38 smithi107 ceph-mon[29404]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:38 smithi107 ceph-mon[40355]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:38.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:38 smithi162 ceph-mon[33228]: pgmap v21: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:40.326 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:40 smithi107 ceph-mon[29404]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:40.327 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:40 smithi107 ceph-mon[40355]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:40.348 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626450 got 180388626454 for osd.6 2024-09-10T18:54:40.348 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:40.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:40 smithi162 ceph-mon[33228]: pgmap v22: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:40.989 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430223 got 206158430227 for osd.7 2024-09-10T18:54:40.989 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:41.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:54:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366954378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe0106e60 msgr2=0x7f3fe0109240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366954378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe0106e60 0x7f3fe0109240 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009d20 tx=0x7f3fd8004c50 comp rx=0 tx=0).stop 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366954378 shutdown_connections 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366954378 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe0106e60 0x7f3fe0109240 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366954378 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 0x7f3fe0106890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366954378 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe00fd8b0 0x7f3fe00fdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366954378 >> 172.21.15.107:0/1366954378 conn(0x7f3fe0071c00 msgr2=0x7f3fe0072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:41.825 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366954378 shutdown_connections 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.768+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366954378 wait complete. 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 Processor -- start 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 -- start start 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 0x7f3fe00fbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 0x7f3fe00fc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe011b000 con 0x7f3fe0106e60 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe011b170 con 0x7f3fe00fe250 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe7126640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe011b330 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.769+0000 7f3fe4ec2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:54548/0 (socket says 172.21.15.107:54548) 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 -- 172.21.15.107:0/1366071758 learned_addr learned my addr 172.21.15.107:0/1366071758 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:41.826 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fdffff640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 0x7f3fe00fbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 -- 172.21.15.107:0/1366071758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 msgr2=0x7f3fe00fbc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 0x7f3fe00fbc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 -- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 msgr2=0x7f3fe00fc160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe56c3640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 0x7f3fe00fc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 0x7f3fe00fc160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 -- 172.21.15.107:0/1366071758 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd4005040 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe56c3640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 0x7f3fe00fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe4ec2640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009550 tx=0x7f3fd4009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd400e930 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd80098b0 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f3fd400f040 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4013b10 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.827 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.770+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe011b9a0 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.771+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe0065b20 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.772+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f3fd400ea90 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.772+0000 7f3fdd7fa640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 0x7f3fd00713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.772+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f3fd408e8f0 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.772+0000 7f3fdffff640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 0x7f3fd00713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.773+0000 7f3fdffff640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 0x7f3fd00713c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00fcdc0 tx=0x7f3fcc00b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:41.828 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.776+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3fd405b270 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.939+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f3fe010bb30 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.941 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.939+0000 7f3fdd7fa640 1 -- 172.21.15.107:0/1366071758 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7f3fd405ac10 con 0x7f3fe00fd8b0 2024-09-10T18:54:41.941 INFO:teuthology.orchestra.run.smithi107.stdout:128849018914 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 msgr2=0x7f3fd00713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 0x7f3fd00713c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00fcdc0 tx=0x7f3fcc00b040 comp rx=0 tx=0).stop 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 msgr2=0x7f3fe00fb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3fd4009550 tx=0x7f3fd4009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 shutdown_connections 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3fd006ef10 0x7f3fd00713c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3fe0106e60 0x7f3fe00fc160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3fe00fe250 0x7f3fe00fbc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.943 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 --2- 172.21.15.107:0/1366071758 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3fe00fd8b0 0x7f3fe00fb6e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 >> 172.21.15.107:0/1366071758 conn(0x7f3fe0071c00 msgr2=0x7f3fe00fa430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 shutdown_connections 2024-09-10T18:54:41.944 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:41.941+0000 7f3fe7126640 1 -- 172.21.15.107:0/1366071758 wait complete. 2024-09-10T18:54:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:42 smithi162 ceph-mon[33228]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1366071758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:54:42.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:42.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:42 smithi107 ceph-mon[29404]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:42.684 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1366071758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:54:42.684 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:42 smithi107 ceph-mon[40355]: pgmap v23: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:42.684 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1366071758' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-09-10T18:54:43.052 INFO:tasks.cephadm.ceph_manager.ceph:need seq 128849018909 got 128849018914 for osd.4 2024-09-10T18:54:43.053 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:43.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:43.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:43.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 -- 172.21.15.107:0/1422152907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fc8a0 msgr2=0x7fb6440fcd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/1422152907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fc8a0 0x7fb6440fcd10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fb638004980 tx=0x7fb638009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 -- 172.21.15.107:0/1422152907 shutdown_connections 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/1422152907 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fd250 0x7fb644109670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/1422152907 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fc8a0 0x7fb6440fcd10 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/1422152907 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 0x7fb644102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 -- 172.21.15.107:0/1422152907 >> 172.21.15.107:0/1422152907 conn(0x7fb6440f8400 msgr2=0x7fb6440fa810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 -- 172.21.15.107:0/1422152907 shutdown_connections 2024-09-10T18:54:43.707 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.705+0000 7fb64ac6c640 1 -- 172.21.15.107:0/1422152907 wait complete. 2024-09-10T18:54:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.706+0000 7fb64ac6c640 1 Processor -- start 2024-09-10T18:54:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.706+0000 7fb64ac6c640 1 -- start start 2024-09-10T18:54:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.706+0000 7fb64ac6c640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 0x7fb644112c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.706+0000 7fb64ac6c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:43.709 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.706+0000 7fb64ac6c640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 0x7fb64410cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb64ac6c640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410d440 con 0x7fb6440fd250 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb64ac6c640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410d5b0 con 0x7fb6441028c0 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb64ac6c640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb64410d770 con 0x7fb6440fc8a0 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb648a08640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 0x7fb644112c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:37618/0 (socket says 172.21.15.107:37618) 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 -- 172.21.15.107:0/2132725039 learned_addr learned my addr 172.21.15.107:0/2132725039 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb649209640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 0x7fb64410cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 -- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 msgr2=0x7fb644112c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 0x7fb644112c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 -- 172.21.15.107:0/2132725039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 msgr2=0x7fb64410cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 0x7fb64410cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.710 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 -- 172.21.15.107:0/2132725039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb638005040 con 0x7fb6440fd250 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb648a08640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 0x7fb644112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb643fff640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb63802a4c0 tx=0x7fb638035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63802ab50 con 0x7fb6440fd250 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fb63802acb0 con 0x7fb6440fd250 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb64410d9f0 con 0x7fb6440fd250 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb63803c410 con 0x7fb6440fd250 2024-09-10T18:54:43.711 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.707+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb64406fb60 con 0x7fb6440fd250 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.709+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6440fdfe0 con 0x7fb6440fd250 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.709+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fb638030020 con 0x7fb6440fd250 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.709+0000 7fb6417fa640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 0x7fb62c0713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.709+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fb6380aeb00 con 0x7fb6440fd250 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.710+0000 7fb648a08640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 0x7fb62c0713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:43.712 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.710+0000 7fb648a08640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 0x7fb62c0713c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb634007fd0 tx=0x7fb634009040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:43.715 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.714+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fb63807b620 con 0x7fb6440fd250 2024-09-10T18:54:43.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.871+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7fb64410e2d0 con 0x7fb6440fd250 2024-09-10T18:54:43.872 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.871+0000 7fb6417fa640 1 -- 172.21.15.107:0/2132725039 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+13 (secure 0 0 0) 0x7fb63807afc0 con 0x7fb6440fd250 2024-09-10T18:54:43.873 INFO:teuthology.orchestra.run.smithi107.stdout:154618822687 2024-09-10T18:54:43.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 msgr2=0x7fb62c0713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:43.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 0x7fb62c0713c0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb634007fd0 tx=0x7fb634009040 comp rx=0 tx=0).stop 2024-09-10T18:54:43.874 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 msgr2=0x7fb6441131a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fb63802a4c0 tx=0x7fb638035040 comp rx=0 tx=0).stop 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 shutdown_connections 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fb62c06ef10 0x7fb62c0713c0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fb6441028c0 0x7fb64410cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fb6440fd250 0x7fb6441131a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 --2- 172.21.15.107:0/2132725039 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fb6440fc8a0 0x7fb644112c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 >> 172.21.15.107:0/2132725039 conn(0x7fb6440f8400 msgr2=0x7fb6440f9da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 shutdown_connections 2024-09-10T18:54:43.875 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:43.873+0000 7fb64ac6c640 1 -- 172.21.15.107:0/2132725039 wait complete. 2024-09-10T18:54:44.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:44 smithi107 ceph-mon[40355]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:44.509 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2132725039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:54:44.509 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:44 smithi107 ceph-mon[29404]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:44.510 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2132725039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:54:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:44 smithi162 ceph-mon[33228]: pgmap v24: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2132725039' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-09-10T18:54:45.143 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822682 got 154618822687 for osd.5 2024-09-10T18:54:45.143 DEBUG:teuthology.parallel:result is None 2024-09-10T18:54:45.143 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-10T18:54:45.143 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:54:46.037 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:46.431 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:46 smithi107 ceph-mon[29404]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:46.431 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:46.431 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:46 smithi107 ceph-mon[40355]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:46.431 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:46 smithi162 ceph-mon[33228]: pgmap v25: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4020192946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fb800 msgr2=0x7fdf380fbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4020192946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fb800 0x7fdf380fbc70 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fdf24004980 tx=0x7fdf24009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4020192946 shutdown_connections 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4020192946 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf380fc1b0 0x7fdf38108580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4020192946 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fb800 0x7fdf380fbc70 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4020192946 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf381017d0 0x7fdf38101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4020192946 >> 172.21.15.107:0/4020192946 conn(0x7fdf38071c00 msgr2=0x7fdf38072000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4020192946 shutdown_connections 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4020192946 wait complete. 2024-09-10T18:54:48.352 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.314+0000 7fdf3f1b9640 1 Processor -- start 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 -- start start 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf380fb800 0x7fdf3810ddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 0x7fdf3810e850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38119000 con 0x7fdf381017d0 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38119170 con 0x7fdf380fb800 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3f1b9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf38119330 con 0x7fdf380fc1b0 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf37fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 0x7fdf3810e850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf37fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:46068/0 (socket says 172.21.15.107:46068) 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf37fff640 1 -- 172.21.15.107:0/4255698096 learned_addr learned my addr 172.21.15.107:0/4255698096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3cf55640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf380fb800 0x7fdf3810ddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:48.353 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 -- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 msgr2=0x7fdf3810e310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 -- 172.21.15.107:0/4255698096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf380fb800 msgr2=0x7fdf3810ddd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf380fb800 0x7fdf3810ddd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf3d756640 1 -- 172.21.15.107:0/4255698096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf300098b0 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.315+0000 7fdf37fff640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf3d756640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 0x7fdf3810e850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf30001da0 tx=0x7fdf30004c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf3000fdd0 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fdf30007cd0 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf24005040 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf30009610 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.316+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf381199a0 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.317+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf380fcef0 con 0x7fdf381017d0 2024-09-10T18:54:48.354 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.317+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fdf30007890 con 0x7fdf381017d0 2024-09-10T18:54:48.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.318+0000 7fdf357fa640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 0x7fdf280713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:48.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.318+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fdf3008e890 con 0x7fdf381017d0 2024-09-10T18:54:48.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.318+0000 7fdf3cf55640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 0x7fdf280713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:48.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.318+0000 7fdf3cf55640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 0x7fdf280713c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c004eb0 tx=0x7fdf2c00b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:48.355 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.322+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fdf3005b340 con 0x7fdf381017d0 2024-09-10T18:54:48.477 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.476+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 --> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdf3810aa30 con 0x7fdf2806ef10 2024-09-10T18:54:48.487 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.486+0000 7fdf357fa640 1 -- 172.21.15.107:0/4255698096 <== mgr.14559 v2:172.21.15.107:6800/339792131 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383565 (secure 0 0 0) 0x7fdf3810aa30 con 0x7fdf2806ef10 2024-09-10T18:54:48.487 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:48 smithi107 ceph-mon[29404]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:48.488 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:48 smithi107 ceph-mon[40355]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:48.491 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:54:48.493 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 msgr2=0x7fdf280713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 0x7fdf280713c0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fdf2c004eb0 tx=0x7fdf2c00b040 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 msgr2=0x7fdf3810e850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 0x7fdf3810e850 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fdf30001da0 tx=0x7fdf30004c50 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 shutdown_connections 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fdf2806ef10 0x7fdf280713c0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fdf381017d0 0x7fdf3810e850 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fdf380fc1b0 0x7fdf3810e310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 --2- 172.21.15.107:0/4255698096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fdf380fb800 0x7fdf3810ddd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 >> 172.21.15.107:0/4255698096 conn(0x7fdf38071c00 msgr2=0x7fdf380fa340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 shutdown_connections 2024-09-10T18:54:48.496 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:48.489+0000 7fdf3f1b9640 1 -- 172.21.15.107:0/4255698096 wait complete. 2024-09-10T18:54:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:48 smithi162 ceph-mon[33228]: pgmap v26: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:48.896 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:48 smithi107 podman[76944]: 2024-09-10 18:54:48.705440975 +0000 UTC m=+0.599032289 container create 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:54:49.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[29404]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:49.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.252 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[40355]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.253 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:48 smithi107 podman[76944]: 2024-09-10 18:54:48.99982957 +0000 UTC m=+0.893420839 container init 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:54:49.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 podman[76944]: 2024-09-10 18:54:49.004917328 +0000 UTC m=+0.898508598 container start 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:54:49.253 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.017Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.017Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.018Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.107 port=9094 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.019Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.043Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.043Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.046Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:49.046Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 bash[76944]: 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 2024-09-10T18:54:49.254 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:49 smithi107 systemd[1]: Started Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:54:49.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:49 smithi162 ceph-mon[33228]: from='client.14694 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:49.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE Bus STOPPING 2024-09-10T18:54:49.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:54:49.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE Bus STOPPED 2024-09-10T18:54:49.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE Bus STARTING 2024-09-10T18:54:49.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE Serving on http://:::9283 2024-09-10T18:54:49.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:54:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:54:49] ENGINE Bus STARTED 2024-09-10T18:54:49.805 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2024-09-10T18:54:47.039841+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":927,"num_read_kb":784,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385876,"kb_used_data":7036,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575340,"statfs":{"total":767960285184,"available":765517148160,"internally_reserved":0,"allocated":7204864,"data_stored":3992702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002765"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434474+0000","last_change":"2024-09-10T18:53:30.496366+0000","last_active":"2024-09-10T18:53:37.434474+0000","last_peered":"2024-09-10T18:53:37.434474+0000","last_clean":"2024-09-10T18:53:37.434474+0000","last_became_active":"2024-09-10T18:53:30.495541+0000","last_became_peered":"2024-09-10T18:53:30.495541+0000","last_unstale":"2024-09-10T18:53:37.434474+0000","last_undegraded":"2024-09-10T18:53:37.434474+0000","last_fullsized":"2024-09-10T18:53:37.434474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:09.551565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151354+0000","last_change":"2024-09-10T18:53:24.321406+0000","last_active":"2024-09-10T18:53:37.151354+0000","last_peered":"2024-09-10T18:53:37.151354+0000","last_clean":"2024-09-10T18:53:37.151354+0000","last_became_active":"2024-09-10T18:53:24.321125+0000","last_became_peered":"2024-09-10T18:53:24.321125+0000","last_unstale":"2024-09-10T18:53:37.151354+0000","last_undegraded":"2024-09-10T18:53:37.151354+0000","last_fullsized":"2024-09-10T18:53:37.151354+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:46.179922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434324+0000","last_change":"2024-09-10T18:53:26.298914+0000","last_active":"2024-09-10T18:53:37.434324+0000","last_peered":"2024-09-10T18:53:37.434324+0000","last_clean":"2024-09-10T18:53:37.434324+0000","last_became_active":"2024-09-10T18:53:26.298589+0000","last_became_peered":"2024-09-10T18:53:26.298589+0000","last_unstale":"2024-09-10T18:53:37.434324+0000","last_undegraded":"2024-09-10T18:53:37.434324+0000","last_fullsized":"2024-09-10T18:53:37.434324+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:06.627755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.147977+0000","last_change":"2024-09-10T18:53:28.305555+0000","last_active":"2024-09-10T18:53:37.147977+0000","last_peered":"2024-09-10T18:53:37.147977+0000","last_clean":"2024-09-10T18:53:37.147977+0000","last_became_active":"2024-09-10T18:53:28.305184+0000","last_became_peered":"2024-09-10T18:53:28.305184+0000","last_unstale":"2024-09-10T18:53:37.147977+0000","last_undegraded":"2024-09-10T18:53:37.147977+0000","last_fullsized":"2024-09-10T18:53:37.147977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:50.501427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435255+0000","last_change":"2024-09-10T18:53:24.312553+0000","last_active":"2024-09-10T18:53:37.435255+0000","last_peered":"2024-09-10T18:53:37.435255+0000","last_clean":"2024-09-10T18:53:37.435255+0000","last_became_active":"2024-09-10T18:53:24.312219+0000","last_became_peered":"2024-09-10T18:53:24.312219+0000","last_unstale":"2024-09-10T18:53:37.435255+0000","last_undegraded":"2024-09-10T18:53:37.435255+0000","last_fullsized":"2024-09-10T18:53:37.435255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:35.218940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.147993+0000","last_change":"2024-09-10T18:53:26.298649+0000","last_active":"2024-09-10T18:53:37.147993+0000","last_peered":"2024-09-10T18:53:37.147993+0000","last_clean":"2024-09-10T18:53:37.147993+0000","last_became_active":"2024-09-10T18:53:26.298353+0000","last_became_peered":"2024-09-10T18:53:26.298353+0000","last_unstale":"2024-09-10T18:53:37.147993+0000","last_undegraded":"2024-09-10T18:53:37.147993+0000","last_fullsized":"2024-09-10T18:53:37.147993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:23.749365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149705+0000","last_change":"2024-09-10T18:53:28.299218+0000","last_active":"2024-09-10T18:53:37.149705+0000","last_peered":"2024-09-10T18:53:37.149705+0000","last_clean":"2024-09-10T18:53:37.149705+0000","last_became_active":"2024-09-10T18:53:28.298021+0000","last_became_peered":"2024-09-10T18:53:28.298021+0000","last_unstale":"2024-09-10T18:53:37.149705+0000","last_undegraded":"2024-09-10T18:53:37.149705+0000","last_fullsized":"2024-09-10T18:53:37.149705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:04.217092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148116+0000","last_change":"2024-09-10T18:53:30.316227+0000","last_active":"2024-09-10T18:53:37.148116+0000","last_peered":"2024-09-10T18:53:37.148116+0000","last_clean":"2024-09-10T18:53:37.148116+0000","last_became_active":"2024-09-10T18:53:30.315935+0000","last_became_peered":"2024-09-10T18:53:30.315935+0000","last_unstale":"2024-09-10T18:53:37.148116+0000","last_undegraded":"2024-09-10T18:53:37.148116+0000","last_fullsized":"2024-09-10T18:53:37.148116+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:24.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435294+0000","last_change":"2024-09-10T18:53:24.323330+0000","last_active":"2024-09-10T18:53:37.435294+0000","last_peered":"2024-09-10T18:53:37.435294+0000","last_clean":"2024-09-10T18:53:37.435294+0000","last_became_active":"2024-09-10T18:53:24.323050+0000","last_became_peered":"2024-09-10T18:53:24.323050+0000","last_unstale":"2024-09-10T18:53:37.435294+0000","last_undegraded":"2024-09-10T18:53:37.435294+0000","last_fullsized":"2024-09-10T18:53:37.435294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:37.847644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148222+0000","last_change":"2024-09-10T18:53:26.294016+0000","last_active":"2024-09-10T18:53:37.148222+0000","last_peered":"2024-09-10T18:53:37.148222+0000","last_clean":"2024-09-10T18:53:37.148222+0000","last_became_active":"2024-09-10T18:53:26.293510+0000","last_became_peered":"2024-09-10T18:53:26.293510+0000","last_unstale":"2024-09-10T18:53:37.148222+0000","last_undegraded":"2024-09-10T18:53:37.148222+0000","last_fullsized":"2024-09-10T18:53:37.148222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:54.773423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435048+0000","last_change":"2024-09-10T18:53:28.296981+0000","last_active":"2024-09-10T18:53:37.435048+0000","last_peered":"2024-09-10T18:53:37.435048+0000","last_clean":"2024-09-10T18:53:37.435048+0000","last_became_active":"2024-09-10T18:53:28.296682+0000","last_became_peered":"2024-09-10T18:53:28.296682+0000","last_unstale":"2024-09-10T18:53:37.435048+0000","last_undegraded":"2024-09-10T18:53:37.435048+0000","last_fullsized":"2024-09-10T18:53:37.435048+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:57.859622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434320+0000","last_change":"2024-09-10T18:53:30.318079+0000","last_active":"2024-09-10T18:53:37.434320+0000","last_peered":"2024-09-10T18:53:37.434320+0000","last_clean":"2024-09-10T18:53:37.434320+0000","last_became_active":"2024-09-10T18:53:30.317459+0000","last_became_peered":"2024-09-10T18:53:30.317459+0000","last_unstale":"2024-09-10T18:53:37.434320+0000","last_undegraded":"2024-09-10T18:53:37.434320+0000","last_fullsized":"2024-09-10T18:53:37.434320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:21.154537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435704+0000","last_change":"2024-09-10T18:53:24.320047+0000","last_active":"2024-09-10T18:53:37.435704+0000","last_peered":"2024-09-10T18:53:37.435704+0000","last_clean":"2024-09-10T18:53:37.435704+0000","last_became_active":"2024-09-10T18:53:24.319698+0000","last_became_peered":"2024-09-10T18:53:24.319698+0000","last_unstale":"2024-09-10T18:53:37.435704+0000","last_undegraded":"2024-09-10T18:53:37.435704+0000","last_fullsized":"2024-09-10T18:53:37.435704+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:06.255321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148340+0000","last_change":"2024-09-10T18:53:26.298680+0000","last_active":"2024-09-10T18:53:37.148340+0000","last_peered":"2024-09-10T18:53:37.148340+0000","last_clean":"2024-09-10T18:53:37.148340+0000","last_became_active":"2024-09-10T18:53:26.298370+0000","last_became_peered":"2024-09-10T18:53:26.298370+0000","last_unstale":"2024-09-10T18:53:37.148340+0000","last_undegraded":"2024-09-10T18:53:37.148340+0000","last_fullsized":"2024-09-10T18:53:37.148340+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:14.386955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434677+0000","last_change":"2024-09-10T18:53:28.303526+0000","last_active":"2024-09-10T18:53:37.434677+0000","last_peered":"2024-09-10T18:53:37.434677+0000","last_clean":"2024-09-10T18:53:37.434677+0000","last_became_active":"2024-09-10T18:53:28.303211+0000","last_became_peered":"2024-09-10T18:53:28.303211+0000","last_unstale":"2024-09-10T18:53:37.434677+0000","last_undegraded":"2024-09-10T18:53:37.434677+0000","last_fullsized":"2024-09-10T18:53:37.434677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:39.302350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150968+0000","last_change":"2024-09-10T18:53:30.316863+0000","last_active":"2024-09-10T18:53:37.150968+0000","last_peered":"2024-09-10T18:53:37.150968+0000","last_clean":"2024-09-10T18:53:37.150968+0000","last_became_active":"2024-09-10T18:53:30.316591+0000","last_became_peered":"2024-09-10T18:53:30.316591+0000","last_unstale":"2024-09-10T18:53:37.150968+0000","last_undegraded":"2024-09-10T18:53:37.150968+0000","last_fullsized":"2024-09-10T18:53:37.150968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:18.392905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148811+0000","last_change":"2024-09-10T18:53:26.296532+0000","last_active":"2024-09-10T18:53:37.148811+0000","last_peered":"2024-09-10T18:53:37.148811+0000","last_clean":"2024-09-10T18:53:37.148811+0000","last_became_active":"2024-09-10T18:53:26.296094+0000","last_became_peered":"2024-09-10T18:53:26.296094+0000","last_unstale":"2024-09-10T18:53:37.148811+0000","last_undegraded":"2024-09-10T18:53:37.148811+0000","last_fullsized":"2024-09-10T18:53:37.148811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:08.669913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435099+0000","last_change":"2024-09-10T18:53:24.322438+0000","last_active":"2024-09-10T18:53:37.435099+0000","last_peered":"2024-09-10T18:53:37.435099+0000","last_clean":"2024-09-10T18:53:37.435099+0000","last_became_active":"2024-09-10T18:53:24.322159+0000","last_became_peered":"2024-09-10T18:53:24.322159+0000","last_unstale":"2024-09-10T18:53:37.435099+0000","last_undegraded":"2024-09-10T18:53:37.435099+0000","last_fullsized":"2024-09-10T18:53:37.435099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:14.283289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149895+0000","last_change":"2024-09-10T18:53:28.300445+0000","last_active":"2024-09-10T18:53:37.149895+0000","last_peered":"2024-09-10T18:53:37.149895+0000","last_clean":"2024-09-10T18:53:37.149895+0000","last_became_active":"2024-09-10T18:53:28.300177+0000","last_became_peered":"2024-09-10T18:53:28.300177+0000","last_unstale":"2024-09-10T18:53:37.149895+0000","last_undegraded":"2024-09-10T18:53:37.149895+0000","last_fullsized":"2024-09-10T18:53:37.149895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:07.545834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434853+0000","last_change":"2024-09-10T18:53:30.322204+0000","last_active":"2024-09-10T18:53:37.434853+0000","last_peered":"2024-09-10T18:53:37.434853+0000","last_clean":"2024-09-10T18:53:37.434853+0000","last_became_active":"2024-09-10T18:53:30.321833+0000","last_became_peered":"2024-09-10T18:53:30.321833+0000","last_unstale":"2024-09-10T18:53:37.434853+0000","last_undegraded":"2024-09-10T18:53:37.434853+0000","last_fullsized":"2024-09-10T18:53:37.434853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:54.063401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148532+0000","last_change":"2024-09-10T18:53:26.297344+0000","last_active":"2024-09-10T18:53:37.148532+0000","last_peered":"2024-09-10T18:53:37.148532+0000","last_clean":"2024-09-10T18:53:37.148532+0000","last_became_active":"2024-09-10T18:53:26.296980+0000","last_became_peered":"2024-09-10T18:53:26.296980+0000","last_unstale":"2024-09-10T18:53:37.148532+0000","last_undegraded":"2024-09-10T18:53:37.148532+0000","last_fullsized":"2024-09-10T18:53:37.148532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:36.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150580+0000","last_change":"2024-09-10T18:53:24.287714+0000","last_active":"2024-09-10T18:53:37.150580+0000","last_peered":"2024-09-10T18:53:37.150580+0000","last_clean":"2024-09-10T18:53:37.150580+0000","last_became_active":"2024-09-10T18:53:24.287418+0000","last_became_peered":"2024-09-10T18:53:24.287418+0000","last_unstale":"2024-09-10T18:53:37.150580+0000","last_undegraded":"2024-09-10T18:53:37.150580+0000","last_fullsized":"2024-09-10T18:53:37.150580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:01.629263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542657+0000","last_change":"2024-09-10T18:53:28.306463+0000","last_active":"2024-09-10T18:54:37.542657+0000","last_peered":"2024-09-10T18:54:37.542657+0000","last_clean":"2024-09-10T18:54:37.542657+0000","last_became_active":"2024-09-10T18:53:28.305935+0000","last_became_peered":"2024-09-10T18:53:28.305935+0000","last_unstale":"2024-09-10T18:54:37.542657+0000","last_undegraded":"2024-09-10T18:54:37.542657+0000","last_fullsized":"2024-09-10T18:54:37.542657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:46.561887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148614+0000","last_change":"2024-09-10T18:53:30.318997+0000","last_active":"2024-09-10T18:53:37.148614+0000","last_peered":"2024-09-10T18:53:37.148614+0000","last_clean":"2024-09-10T18:53:37.148614+0000","last_became_active":"2024-09-10T18:53:30.318735+0000","last_became_peered":"2024-09-10T18:53:30.318735+0000","last_unstale":"2024-09-10T18:53:37.148614+0000","last_undegraded":"2024-09-10T18:53:37.148614+0000","last_fullsized":"2024-09-10T18:53:37.148614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:16.108717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150194+0000","last_change":"2024-09-10T18:53:26.301372+0000","last_active":"2024-09-10T18:53:37.150194+0000","last_peered":"2024-09-10T18:53:37.150194+0000","last_clean":"2024-09-10T18:53:37.150194+0000","last_became_active":"2024-09-10T18:53:26.300708+0000","last_became_peered":"2024-09-10T18:53:26.300708+0000","last_unstale":"2024-09-10T18:53:37.150194+0000","last_undegraded":"2024-09-10T18:53:37.150194+0000","last_fullsized":"2024-09-10T18:53:37.150194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:00.455320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"52'1","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149798+0000","last_change":"2024-09-10T18:53:24.287264+0000","last_active":"2024-09-10T18:53:37.149798+0000","last_peered":"2024-09-10T18:53:37.149798+0000","last_clean":"2024-09-10T18:53:37.149798+0000","last_became_active":"2024-09-10T18:53:24.286916+0000","last_became_peered":"2024-09-10T18:53:24.286916+0000","last_unstale":"2024-09-10T18:53:37.149798+0000","last_undegraded":"2024-09-10T18:53:37.149798+0000","last_fullsized":"2024-09-10T18:53:37.149798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.363101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543082+0000","last_change":"2024-09-10T18:53:28.307959+0000","last_active":"2024-09-10T18:54:37.543082+0000","last_peered":"2024-09-10T18:54:37.543082+0000","last_clean":"2024-09-10T18:54:37.543082+0000","last_became_active":"2024-09-10T18:53:28.307578+0000","last_became_peered":"2024-09-10T18:53:28.307578+0000","last_unstale":"2024-09-10T18:54:37.543082+0000","last_undegraded":"2024-09-10T18:54:37.543082+0000","last_fullsized":"2024-09-10T18:54:37.543082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:11.692163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434467+0000","last_change":"2024-09-10T18:53:30.322430+0000","last_active":"2024-09-10T18:53:37.434467+0000","last_peered":"2024-09-10T18:53:37.434467+0000","last_clean":"2024-09-10T18:53:37.434467+0000","last_became_active":"2024-09-10T18:53:30.322136+0000","last_became_peered":"2024-09-10T18:53:30.322136+0000","last_unstale":"2024-09-10T18:53:37.434467+0000","last_undegraded":"2024-09-10T18:53:37.434467+0000","last_fullsized":"2024-09-10T18:53:37.434467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:11.215386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148726+0000","last_change":"2024-09-10T18:53:26.296824+0000","last_active":"2024-09-10T18:53:37.148726+0000","last_peered":"2024-09-10T18:53:37.148726+0000","last_clean":"2024-09-10T18:53:37.148726+0000","last_became_active":"2024-09-10T18:53:26.296378+0000","last_became_peered":"2024-09-10T18:53:26.296378+0000","last_unstale":"2024-09-10T18:53:37.148726+0000","last_undegraded":"2024-09-10T18:53:37.148726+0000","last_fullsized":"2024-09-10T18:53:37.148726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:53.035182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434853+0000","last_change":"2024-09-10T18:53:24.323809+0000","last_active":"2024-09-10T18:53:37.434853+0000","last_peered":"2024-09-10T18:53:37.434853+0000","last_clean":"2024-09-10T18:53:37.434853+0000","last_became_active":"2024-09-10T18:53:24.323101+0000","last_became_peered":"2024-09-10T18:53:24.323101+0000","last_unstale":"2024-09-10T18:53:37.434853+0000","last_undegraded":"2024-09-10T18:53:37.434853+0000","last_fullsized":"2024-09-10T18:53:37.434853+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:44.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435720+0000","last_change":"2024-09-10T18:53:28.302549+0000","last_active":"2024-09-10T18:53:37.435720+0000","last_peered":"2024-09-10T18:53:37.435720+0000","last_clean":"2024-09-10T18:53:37.435720+0000","last_became_active":"2024-09-10T18:53:28.301984+0000","last_became_peered":"2024-09-10T18:53:28.301984+0000","last_unstale":"2024-09-10T18:53:37.435720+0000","last_undegraded":"2024-09-10T18:53:37.435720+0000","last_fullsized":"2024-09-10T18:53:37.435720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:15.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434751+0000","last_change":"2024-09-10T18:53:30.493864+0000","last_active":"2024-09-10T18:53:37.434751+0000","last_peered":"2024-09-10T18:53:37.434751+0000","last_clean":"2024-09-10T18:53:37.434751+0000","last_became_active":"2024-09-10T18:53:30.493550+0000","last_became_peered":"2024-09-10T18:53:30.493550+0000","last_unstale":"2024-09-10T18:53:37.434751+0000","last_undegraded":"2024-09-10T18:53:37.434751+0000","last_fullsized":"2024-09-10T18:53:37.434751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:38.229848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148178+0000","last_change":"2024-09-10T18:53:26.298164+0000","last_active":"2024-09-10T18:53:37.148178+0000","last_peered":"2024-09-10T18:53:37.148178+0000","last_clean":"2024-09-10T18:53:37.148178+0000","last_became_active":"2024-09-10T18:53:26.297438+0000","last_became_peered":"2024-09-10T18:53:26.297438+0000","last_unstale":"2024-09-10T18:53:37.148178+0000","last_undegraded":"2024-09-10T18:53:37.148178+0000","last_fullsized":"2024-09-10T18:53:37.148178+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:33.416772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363534+0000","last_change":"2024-09-10T18:53:24.318909+0000","last_active":"2024-09-10T18:53:37.363534+0000","last_peered":"2024-09-10T18:53:37.363534+0000","last_clean":"2024-09-10T18:53:37.363534+0000","last_became_active":"2024-09-10T18:53:24.318518+0000","last_became_peered":"2024-09-10T18:53:24.318518+0000","last_unstale":"2024-09-10T18:53:37.363534+0000","last_undegraded":"2024-09-10T18:53:37.363534+0000","last_fullsized":"2024-09-10T18:53:37.363534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:53.887357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542695+0000","last_change":"2024-09-10T18:53:28.311491+0000","last_active":"2024-09-10T18:54:37.542695+0000","last_peered":"2024-09-10T18:54:37.542695+0000","last_clean":"2024-09-10T18:54:37.542695+0000","last_became_active":"2024-09-10T18:53:28.310609+0000","last_became_peered":"2024-09-10T18:53:28.310609+0000","last_unstale":"2024-09-10T18:54:37.542695+0000","last_undegraded":"2024-09-10T18:54:37.542695+0000","last_fullsized":"2024-09-10T18:54:37.542695+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:46.705934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434927+0000","last_change":"2024-09-10T18:53:30.494894+0000","last_active":"2024-09-10T18:53:37.434927+0000","last_peered":"2024-09-10T18:53:37.434927+0000","last_clean":"2024-09-10T18:53:37.434927+0000","last_became_active":"2024-09-10T18:53:30.494124+0000","last_became_peered":"2024-09-10T18:53:30.494124+0000","last_unstale":"2024-09-10T18:53:37.434927+0000","last_undegraded":"2024-09-10T18:53:37.434927+0000","last_fullsized":"2024-09-10T18:53:37.434927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:27.871833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434270+0000","last_change":"2024-09-10T18:53:26.436695+0000","last_active":"2024-09-10T18:53:37.434270+0000","last_peered":"2024-09-10T18:53:37.434270+0000","last_clean":"2024-09-10T18:53:37.434270+0000","last_became_active":"2024-09-10T18:53:26.436312+0000","last_became_peered":"2024-09-10T18:53:26.436312+0000","last_unstale":"2024-09-10T18:53:37.434270+0000","last_undegraded":"2024-09-10T18:53:37.434270+0000","last_fullsized":"2024-09-10T18:53:37.434270+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.878924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150028+0000","last_change":"2024-09-10T18:53:24.285419+0000","last_active":"2024-09-10T18:53:37.150028+0000","last_peered":"2024-09-10T18:53:37.150028+0000","last_clean":"2024-09-10T18:53:37.150028+0000","last_became_active":"2024-09-10T18:53:24.285142+0000","last_became_peered":"2024-09-10T18:53:24.285142+0000","last_unstale":"2024-09-10T18:53:37.150028+0000","last_undegraded":"2024-09-10T18:53:37.150028+0000","last_fullsized":"2024-09-10T18:53:37.150028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:46.080848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148054+0000","last_change":"2024-09-10T18:53:28.306671+0000","last_active":"2024-09-10T18:53:37.148054+0000","last_peered":"2024-09-10T18:53:37.148054+0000","last_clean":"2024-09-10T18:53:37.148054+0000","last_became_active":"2024-09-10T18:53:28.306356+0000","last_became_peered":"2024-09-10T18:53:28.306356+0000","last_unstale":"2024-09-10T18:53:37.148054+0000","last_undegraded":"2024-09-10T18:53:37.148054+0000","last_fullsized":"2024-09-10T18:53:37.148054+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:19.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148275+0000","last_change":"2024-09-10T18:53:30.318027+0000","last_active":"2024-09-10T18:53:37.148275+0000","last_peered":"2024-09-10T18:53:37.148275+0000","last_clean":"2024-09-10T18:53:37.148275+0000","last_became_active":"2024-09-10T18:53:30.317760+0000","last_became_peered":"2024-09-10T18:53:30.317760+0000","last_unstale":"2024-09-10T18:53:37.148275+0000","last_undegraded":"2024-09-10T18:53:37.148275+0000","last_fullsized":"2024-09-10T18:53:37.148275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:52.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151530+0000","last_change":"2024-09-10T18:53:26.436852+0000","last_active":"2024-09-10T18:53:37.151530+0000","last_peered":"2024-09-10T18:53:37.151530+0000","last_clean":"2024-09-10T18:53:37.151530+0000","last_became_active":"2024-09-10T18:53:26.436347+0000","last_became_peered":"2024-09-10T18:53:26.436347+0000","last_unstale":"2024-09-10T18:53:37.151530+0000","last_undegraded":"2024-09-10T18:53:37.151530+0000","last_fullsized":"2024-09-10T18:53:37.151530+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:47.996764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435323+0000","last_change":"2024-09-10T18:53:24.323366+0000","last_active":"2024-09-10T18:53:37.435323+0000","last_peered":"2024-09-10T18:53:37.435323+0000","last_clean":"2024-09-10T18:53:37.435323+0000","last_became_active":"2024-09-10T18:53:24.323123+0000","last_became_peered":"2024-09-10T18:53:24.323123+0000","last_unstale":"2024-09-10T18:53:37.435323+0000","last_undegraded":"2024-09-10T18:53:37.435323+0000","last_fullsized":"2024-09-10T18:53:37.435323+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.514253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.362951+0000","last_change":"2024-09-10T18:53:28.303903+0000","last_active":"2024-09-10T18:53:37.362951+0000","last_peered":"2024-09-10T18:53:37.362951+0000","last_clean":"2024-09-10T18:53:37.362951+0000","last_became_active":"2024-09-10T18:53:28.303638+0000","last_became_peered":"2024-09-10T18:53:28.303638+0000","last_unstale":"2024-09-10T18:53:37.362951+0000","last_undegraded":"2024-09-10T18:53:37.362951+0000","last_fullsized":"2024-09-10T18:53:37.362951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:32.062951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149663+0000","last_change":"2024-09-10T18:53:30.493954+0000","last_active":"2024-09-10T18:53:37.149663+0000","last_peered":"2024-09-10T18:53:37.149663+0000","last_clean":"2024-09-10T18:53:37.149663+0000","last_became_active":"2024-09-10T18:53:30.493580+0000","last_became_peered":"2024-09-10T18:53:30.493580+0000","last_unstale":"2024-09-10T18:53:37.149663+0000","last_undegraded":"2024-09-10T18:53:37.149663+0000","last_fullsized":"2024-09-10T18:53:37.149663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:36.616675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150503+0000","last_change":"2024-09-10T18:53:26.301783+0000","last_active":"2024-09-10T18:53:37.150503+0000","last_peered":"2024-09-10T18:53:37.150503+0000","last_clean":"2024-09-10T18:53:37.150503+0000","last_became_active":"2024-09-10T18:53:26.301155+0000","last_became_peered":"2024-09-10T18:53:26.301155+0000","last_unstale":"2024-09-10T18:53:37.150503+0000","last_undegraded":"2024-09-10T18:53:37.150503+0000","last_fullsized":"2024-09-10T18:53:37.150503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:38.192190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150234+0000","last_change":"2024-09-10T18:53:24.288264+0000","last_active":"2024-09-10T18:53:37.150234+0000","last_peered":"2024-09-10T18:53:37.150234+0000","last_clean":"2024-09-10T18:53:37.150234+0000","last_became_active":"2024-09-10T18:53:24.287980+0000","last_became_peered":"2024-09-10T18:53:24.287980+0000","last_unstale":"2024-09-10T18:53:37.150234+0000","last_undegraded":"2024-09-10T18:53:37.150234+0000","last_fullsized":"2024-09-10T18:53:37.150234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:50.999623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542661+0000","last_change":"2024-09-10T18:53:28.303638+0000","last_active":"2024-09-10T18:54:37.542661+0000","last_peered":"2024-09-10T18:54:37.542661+0000","last_clean":"2024-09-10T18:54:37.542661+0000","last_became_active":"2024-09-10T18:53:28.303317+0000","last_became_peered":"2024-09-10T18:53:28.303317+0000","last_unstale":"2024-09-10T18:54:37.542661+0000","last_undegraded":"2024-09-10T18:54:37.542661+0000","last_fullsized":"2024-09-10T18:54:37.542661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:59.432446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150765+0000","last_change":"2024-09-10T18:53:30.317428+0000","last_active":"2024-09-10T18:53:37.150765+0000","last_peered":"2024-09-10T18:53:37.150765+0000","last_clean":"2024-09-10T18:53:37.150765+0000","last_became_active":"2024-09-10T18:53:30.317136+0000","last_became_peered":"2024-09-10T18:53:30.317136+0000","last_unstale":"2024-09-10T18:53:37.150765+0000","last_undegraded":"2024-09-10T18:53:37.150765+0000","last_fullsized":"2024-09-10T18:53:37.150765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:09.859951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542656+0000","last_change":"2024-09-10T18:53:26.302071+0000","last_active":"2024-09-10T18:54:37.542656+0000","last_peered":"2024-09-10T18:54:37.542656+0000","last_clean":"2024-09-10T18:54:37.542656+0000","last_became_active":"2024-09-10T18:53:26.301787+0000","last_became_peered":"2024-09-10T18:53:26.301787+0000","last_unstale":"2024-09-10T18:54:37.542656+0000","last_undegraded":"2024-09-10T18:54:37.542656+0000","last_fullsized":"2024-09-10T18:54:37.542656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:52.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434865+0000","last_change":"2024-09-10T18:53:24.315960+0000","last_active":"2024-09-10T18:53:37.434865+0000","last_peered":"2024-09-10T18:53:37.434865+0000","last_clean":"2024-09-10T18:53:37.434865+0000","last_became_active":"2024-09-10T18:53:24.315702+0000","last_became_peered":"2024-09-10T18:53:24.315702+0000","last_unstale":"2024-09-10T18:53:37.434865+0000","last_undegraded":"2024-09-10T18:53:37.434865+0000","last_fullsized":"2024-09-10T18:53:37.434865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:41.956660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151414+0000","last_change":"2024-09-10T18:53:28.304974+0000","last_active":"2024-09-10T18:53:37.151414+0000","last_peered":"2024-09-10T18:53:37.151414+0000","last_clean":"2024-09-10T18:53:37.151414+0000","last_became_active":"2024-09-10T18:53:28.304601+0000","last_became_peered":"2024-09-10T18:53:28.304601+0000","last_unstale":"2024-09-10T18:53:37.151414+0000","last_undegraded":"2024-09-10T18:53:37.151414+0000","last_fullsized":"2024-09-10T18:53:37.151414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:51.997529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435038+0000","last_change":"2024-09-10T18:53:30.319419+0000","last_active":"2024-09-10T18:53:37.435038+0000","last_peered":"2024-09-10T18:53:37.435038+0000","last_clean":"2024-09-10T18:53:37.435038+0000","last_became_active":"2024-09-10T18:53:30.319147+0000","last_became_peered":"2024-09-10T18:53:30.319147+0000","last_unstale":"2024-09-10T18:53:37.435038+0000","last_undegraded":"2024-09-10T18:53:37.435038+0000","last_fullsized":"2024-09-10T18:53:37.435038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:34.252357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150212+0000","last_change":"2024-09-10T18:53:26.303246+0000","last_active":"2024-09-10T18:53:37.150212+0000","last_peered":"2024-09-10T18:53:37.150212+0000","last_clean":"2024-09-10T18:53:37.150212+0000","last_became_active":"2024-09-10T18:53:26.302458+0000","last_became_peered":"2024-09-10T18:53:26.302458+0000","last_unstale":"2024-09-10T18:53:37.150212+0000","last_undegraded":"2024-09-10T18:53:37.150212+0000","last_fullsized":"2024-09-10T18:53:37.150212+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:55.887266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:45.727202+0000","last_change":"2024-09-10T18:53:28.290511+0000","last_active":"2024-09-10T18:54:45.727202+0000","last_peered":"2024-09-10T18:54:45.727202+0000","last_clean":"2024-09-10T18:54:45.727202+0000","last_became_active":"2024-09-10T18:53:24.284144+0000","last_became_peered":"2024-09-10T18:53:24.284144+0000","last_unstale":"2024-09-10T18:54:45.727202+0000","last_undegraded":"2024-09-10T18:54:45.727202+0000","last_fullsized":"2024-09-10T18:54:45.727202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:45.531721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051018899999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434613+0000","last_change":"2024-09-10T18:53:24.289353+0000","last_active":"2024-09-10T18:53:37.434613+0000","last_peered":"2024-09-10T18:53:37.434613+0000","last_clean":"2024-09-10T18:53:37.434613+0000","last_became_active":"2024-09-10T18:53:24.289052+0000","last_became_peered":"2024-09-10T18:53:24.289052+0000","last_unstale":"2024-09-10T18:53:37.434613+0000","last_undegraded":"2024-09-10T18:53:37.434613+0000","last_fullsized":"2024-09-10T18:53:37.434613+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:25.959436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435053+0000","last_change":"2024-09-10T18:53:28.299521+0000","last_active":"2024-09-10T18:53:37.435053+0000","last_peered":"2024-09-10T18:53:37.435053+0000","last_clean":"2024-09-10T18:53:37.435053+0000","last_became_active":"2024-09-10T18:53:28.299223+0000","last_became_peered":"2024-09-10T18:53:28.299223+0000","last_unstale":"2024-09-10T18:53:37.435053+0000","last_undegraded":"2024-09-10T18:53:37.435053+0000","last_fullsized":"2024-09-10T18:53:37.435053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:38.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435061+0000","last_change":"2024-09-10T18:53:30.493741+0000","last_active":"2024-09-10T18:53:37.435061+0000","last_peered":"2024-09-10T18:53:37.435061+0000","last_clean":"2024-09-10T18:53:37.435061+0000","last_became_active":"2024-09-10T18:53:30.493434+0000","last_became_peered":"2024-09-10T18:53:30.493434+0000","last_unstale":"2024-09-10T18:53:37.435061+0000","last_undegraded":"2024-09-10T18:53:37.435061+0000","last_fullsized":"2024-09-10T18:53:37.435061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:04.260417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151607+0000","last_change":"2024-09-10T18:53:26.301162+0000","last_active":"2024-09-10T18:53:37.151607+0000","last_peered":"2024-09-10T18:53:37.151607+0000","last_clean":"2024-09-10T18:53:37.151607+0000","last_became_active":"2024-09-10T18:53:26.300871+0000","last_became_peered":"2024-09-10T18:53:26.300871+0000","last_unstale":"2024-09-10T18:53:37.151607+0000","last_undegraded":"2024-09-10T18:53:37.151607+0000","last_fullsized":"2024-09-10T18:53:37.151607+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:51.606275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151131+0000","last_change":"2024-09-10T18:53:28.299930+0000","last_active":"2024-09-10T18:53:37.151131+0000","last_peered":"2024-09-10T18:53:37.151131+0000","last_clean":"2024-09-10T18:53:37.151131+0000","last_became_active":"2024-09-10T18:53:24.289641+0000","last_became_peered":"2024-09-10T18:53:24.289641+0000","last_unstale":"2024-09-10T18:53:37.151131+0000","last_undegraded":"2024-09-10T18:53:37.151131+0000","last_fullsized":"2024-09-10T18:53:37.151131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:16.222970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002086638,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434722+0000","last_change":"2024-09-10T18:53:24.323365+0000","last_active":"2024-09-10T18:53:37.434722+0000","last_peered":"2024-09-10T18:53:37.434722+0000","last_clean":"2024-09-10T18:53:37.434722+0000","last_became_active":"2024-09-10T18:53:24.323022+0000","last_became_peered":"2024-09-10T18:53:24.323022+0000","last_unstale":"2024-09-10T18:53:37.434722+0000","last_undegraded":"2024-09-10T18:53:37.434722+0000","last_fullsized":"2024-09-10T18:53:37.434722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:09.795084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435804+0000","last_change":"2024-09-10T18:53:28.301432+0000","last_active":"2024-09-10T18:53:37.435804+0000","last_peered":"2024-09-10T18:53:37.435804+0000","last_clean":"2024-09-10T18:53:37.435804+0000","last_became_active":"2024-09-10T18:53:28.301153+0000","last_became_peered":"2024-09-10T18:53:28.301153+0000","last_unstale":"2024-09-10T18:53:37.435804+0000","last_undegraded":"2024-09-10T18:53:37.435804+0000","last_fullsized":"2024-09-10T18:53:37.435804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:06.052714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149816+0000","last_change":"2024-09-10T18:53:30.316591+0000","last_active":"2024-09-10T18:53:37.149816+0000","last_peered":"2024-09-10T18:53:37.149816+0000","last_clean":"2024-09-10T18:53:37.149816+0000","last_became_active":"2024-09-10T18:53:30.316280+0000","last_became_peered":"2024-09-10T18:53:30.316280+0000","last_unstale":"2024-09-10T18:53:37.149816+0000","last_undegraded":"2024-09-10T18:53:37.149816+0000","last_fullsized":"2024-09-10T18:53:37.149816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:02.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363345+0000","last_change":"2024-09-10T18:53:26.435625+0000","last_active":"2024-09-10T18:53:37.363345+0000","last_peered":"2024-09-10T18:53:37.363345+0000","last_clean":"2024-09-10T18:53:37.363345+0000","last_became_active":"2024-09-10T18:53:26.435316+0000","last_became_peered":"2024-09-10T18:53:26.435316+0000","last_unstale":"2024-09-10T18:53:37.363345+0000","last_undegraded":"2024-09-10T18:53:37.363345+0000","last_fullsized":"2024-09-10T18:53:37.363345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:54.666725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434743+0000","last_change":"2024-09-10T18:53:28.302455+0000","last_active":"2024-09-10T18:53:37.434743+0000","last_peered":"2024-09-10T18:53:37.434743+0000","last_clean":"2024-09-10T18:53:37.434743+0000","last_became_active":"2024-09-10T18:53:24.318491+0000","last_became_peered":"2024-09-10T18:53:24.318491+0000","last_unstale":"2024-09-10T18:53:37.434743+0000","last_undegraded":"2024-09-10T18:53:37.434743+0000","last_fullsized":"2024-09-10T18:53:37.434743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:49.320134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012118369999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434877+0000","last_change":"2024-09-10T18:53:24.317399+0000","last_active":"2024-09-10T18:53:37.434877+0000","last_peered":"2024-09-10T18:53:37.434877+0000","last_clean":"2024-09-10T18:53:37.434877+0000","last_became_active":"2024-09-10T18:53:24.316221+0000","last_became_peered":"2024-09-10T18:53:24.316221+0000","last_unstale":"2024-09-10T18:53:37.434877+0000","last_undegraded":"2024-09-10T18:53:37.434877+0000","last_fullsized":"2024-09-10T18:53:37.434877+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:15.626834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:03.080376+0000","last_change":"2024-09-10T18:53:12.962331+0000","last_active":"2024-09-10T18:54:03.080376+0000","last_peered":"2024-09-10T18:54:03.080376+0000","last_clean":"2024-09-10T18:54:03.080376+0000","last_became_active":"2024-09-10T18:53:12.956704+0000","last_became_peered":"2024-09-10T18:53:12.956704+0000","last_unstale":"2024-09-10T18:54:03.080376+0000","last_undegraded":"2024-09-10T18:54:03.080376+0000","last_fullsized":"2024-09-10T18:54:03.080376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:13.632166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:13.632166+0000","last_clean_scrub_stamp":"2024-09-10T18:51:13.632166+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:41.684726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435522+0000","last_change":"2024-09-10T18:53:28.295888+0000","last_active":"2024-09-10T18:53:37.435522+0000","last_peered":"2024-09-10T18:53:37.435522+0000","last_clean":"2024-09-10T18:53:37.435522+0000","last_became_active":"2024-09-10T18:53:28.295573+0000","last_became_peered":"2024-09-10T18:53:28.295573+0000","last_unstale":"2024-09-10T18:53:37.435522+0000","last_undegraded":"2024-09-10T18:53:37.435522+0000","last_fullsized":"2024-09-10T18:53:37.435522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:37.312150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435821+0000","last_change":"2024-09-10T18:53:30.320283+0000","last_active":"2024-09-10T18:53:37.435821+0000","last_peered":"2024-09-10T18:53:37.435821+0000","last_clean":"2024-09-10T18:53:37.435821+0000","last_became_active":"2024-09-10T18:53:30.319653+0000","last_became_peered":"2024-09-10T18:53:30.319653+0000","last_unstale":"2024-09-10T18:53:37.435821+0000","last_undegraded":"2024-09-10T18:53:37.435821+0000","last_fullsized":"2024-09-10T18:53:37.435821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:53.659336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363197+0000","last_change":"2024-09-10T18:53:26.435558+0000","last_active":"2024-09-10T18:53:37.363197+0000","last_peered":"2024-09-10T18:53:37.363197+0000","last_clean":"2024-09-10T18:53:37.363197+0000","last_became_active":"2024-09-10T18:53:26.434916+0000","last_became_peered":"2024-09-10T18:53:26.434916+0000","last_unstale":"2024-09-10T18:53:37.363197+0000","last_undegraded":"2024-09-10T18:53:37.363197+0000","last_fullsized":"2024-09-10T18:53:37.363197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:08.784877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435453+0000","last_change":"2024-09-10T18:53:24.286621+0000","last_active":"2024-09-10T18:53:37.435453+0000","last_peered":"2024-09-10T18:53:37.435453+0000","last_clean":"2024-09-10T18:53:37.435453+0000","last_became_active":"2024-09-10T18:53:24.286301+0000","last_became_peered":"2024-09-10T18:53:24.286301+0000","last_unstale":"2024-09-10T18:53:37.435453+0000","last_undegraded":"2024-09-10T18:53:37.435453+0000","last_fullsized":"2024-09-10T18:53:37.435453+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:53.791835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434854+0000","last_change":"2024-09-10T18:53:28.306824+0000","last_active":"2024-09-10T18:53:37.434854+0000","last_peered":"2024-09-10T18:53:37.434854+0000","last_clean":"2024-09-10T18:53:37.434854+0000","last_became_active":"2024-09-10T18:53:28.306491+0000","last_became_peered":"2024-09-10T18:53:28.306491+0000","last_unstale":"2024-09-10T18:53:37.434854+0000","last_undegraded":"2024-09-10T18:53:37.434854+0000","last_fullsized":"2024-09-10T18:53:37.434854+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:51.444075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434799+0000","last_change":"2024-09-10T18:53:30.324145+0000","last_active":"2024-09-10T18:53:37.434799+0000","last_peered":"2024-09-10T18:53:37.434799+0000","last_clean":"2024-09-10T18:53:37.434799+0000","last_became_active":"2024-09-10T18:53:30.323941+0000","last_became_peered":"2024-09-10T18:53:30.323941+0000","last_unstale":"2024-09-10T18:53:37.434799+0000","last_undegraded":"2024-09-10T18:53:37.434799+0000","last_fullsized":"2024-09-10T18:53:37.434799+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:22.303569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151367+0000","last_change":"2024-09-10T18:53:26.303781+0000","last_active":"2024-09-10T18:53:37.151367+0000","last_peered":"2024-09-10T18:53:37.151367+0000","last_clean":"2024-09-10T18:53:37.151367+0000","last_became_active":"2024-09-10T18:53:26.303330+0000","last_became_peered":"2024-09-10T18:53:26.303330+0000","last_unstale":"2024-09-10T18:53:37.151367+0000","last_undegraded":"2024-09-10T18:53:37.151367+0000","last_fullsized":"2024-09-10T18:53:37.151367+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:26.699302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150382+0000","last_change":"2024-09-10T18:53:24.286522+0000","last_active":"2024-09-10T18:53:37.150382+0000","last_peered":"2024-09-10T18:53:37.150382+0000","last_clean":"2024-09-10T18:53:37.150382+0000","last_became_active":"2024-09-10T18:53:24.286255+0000","last_became_peered":"2024-09-10T18:53:24.286255+0000","last_unstale":"2024-09-10T18:53:37.150382+0000","last_undegraded":"2024-09-10T18:53:37.150382+0000","last_fullsized":"2024-09-10T18:53:37.150382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:31.711842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434726+0000","last_change":"2024-09-10T18:53:28.306325+0000","last_active":"2024-09-10T18:53:37.434726+0000","last_peered":"2024-09-10T18:53:37.434726+0000","last_clean":"2024-09-10T18:53:37.434726+0000","last_became_active":"2024-09-10T18:53:28.305778+0000","last_became_peered":"2024-09-10T18:53:28.305778+0000","last_unstale":"2024-09-10T18:53:37.434726+0000","last_undegraded":"2024-09-10T18:53:37.434726+0000","last_fullsized":"2024-09-10T18:53:37.434726+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:17.756972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150970+0000","last_change":"2024-09-10T18:53:30.317479+0000","last_active":"2024-09-10T18:53:37.150970+0000","last_peered":"2024-09-10T18:53:37.150970+0000","last_clean":"2024-09-10T18:53:37.150970+0000","last_became_active":"2024-09-10T18:53:30.317187+0000","last_became_peered":"2024-09-10T18:53:30.317187+0000","last_unstale":"2024-09-10T18:53:37.150970+0000","last_undegraded":"2024-09-10T18:53:37.150970+0000","last_fullsized":"2024-09-10T18:53:37.150970+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.902323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434316+0000","last_change":"2024-09-10T18:53:26.435709+0000","last_active":"2024-09-10T18:53:37.434316+0000","last_peered":"2024-09-10T18:53:37.434316+0000","last_clean":"2024-09-10T18:53:37.434316+0000","last_became_active":"2024-09-10T18:53:26.435438+0000","last_became_peered":"2024-09-10T18:53:26.435438+0000","last_unstale":"2024-09-10T18:53:37.434316+0000","last_undegraded":"2024-09-10T18:53:37.434316+0000","last_fullsized":"2024-09-10T18:53:37.434316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:40.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435224+0000","last_change":"2024-09-10T18:53:24.287837+0000","last_active":"2024-09-10T18:53:37.435224+0000","last_peered":"2024-09-10T18:53:37.435224+0000","last_clean":"2024-09-10T18:53:37.435224+0000","last_became_active":"2024-09-10T18:53:24.287579+0000","last_became_peered":"2024-09-10T18:53:24.287579+0000","last_unstale":"2024-09-10T18:53:37.435224+0000","last_undegraded":"2024-09-10T18:53:37.435224+0000","last_fullsized":"2024-09-10T18:53:37.435224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:20.965123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543090+0000","last_change":"2024-09-10T18:53:28.298827+0000","last_active":"2024-09-10T18:54:37.543090+0000","last_peered":"2024-09-10T18:54:37.543090+0000","last_clean":"2024-09-10T18:54:37.543090+0000","last_became_active":"2024-09-10T18:53:28.298559+0000","last_became_peered":"2024-09-10T18:53:28.298559+0000","last_unstale":"2024-09-10T18:54:37.543090+0000","last_undegraded":"2024-09-10T18:54:37.543090+0000","last_fullsized":"2024-09-10T18:54:37.543090+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:13.384749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434513+0000","last_change":"2024-09-10T18:53:30.494732+0000","last_active":"2024-09-10T18:53:37.434513+0000","last_peered":"2024-09-10T18:53:37.434513+0000","last_clean":"2024-09-10T18:53:37.434513+0000","last_became_active":"2024-09-10T18:53:30.494397+0000","last_became_peered":"2024-09-10T18:53:30.494397+0000","last_unstale":"2024-09-10T18:53:37.434513+0000","last_undegraded":"2024-09-10T18:53:37.434513+0000","last_fullsized":"2024-09-10T18:53:37.434513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:27.019655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148060+0000","last_change":"2024-09-10T18:53:26.291082+0000","last_active":"2024-09-10T18:53:37.148060+0000","last_peered":"2024-09-10T18:53:37.148060+0000","last_clean":"2024-09-10T18:53:37.148060+0000","last_became_active":"2024-09-10T18:53:26.290878+0000","last_became_peered":"2024-09-10T18:53:26.290878+0000","last_unstale":"2024-09-10T18:53:37.148060+0000","last_undegraded":"2024-09-10T18:53:37.148060+0000","last_fullsized":"2024-09-10T18:53:37.148060+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:43.920371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148921+0000","last_change":"2024-09-10T18:53:24.286439+0000","last_active":"2024-09-10T18:53:37.148921+0000","last_peered":"2024-09-10T18:53:37.148921+0000","last_clean":"2024-09-10T18:53:37.148921+0000","last_became_active":"2024-09-10T18:53:24.286198+0000","last_became_peered":"2024-09-10T18:53:24.286198+0000","last_unstale":"2024-09-10T18:53:37.148921+0000","last_undegraded":"2024-09-10T18:53:37.148921+0000","last_fullsized":"2024-09-10T18:53:37.148921+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:27.304101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434289+0000","last_change":"2024-09-10T18:53:28.302633+0000","last_active":"2024-09-10T18:53:37.434289+0000","last_peered":"2024-09-10T18:53:37.434289+0000","last_clean":"2024-09-10T18:53:37.434289+0000","last_became_active":"2024-09-10T18:53:28.299141+0000","last_became_peered":"2024-09-10T18:53:28.299141+0000","last_unstale":"2024-09-10T18:53:37.434289+0000","last_undegraded":"2024-09-10T18:53:37.434289+0000","last_fullsized":"2024-09-10T18:53:37.434289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:45.400311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434336+0000","last_change":"2024-09-10T18:53:30.319194+0000","last_active":"2024-09-10T18:53:37.434336+0000","last_peered":"2024-09-10T18:53:37.434336+0000","last_clean":"2024-09-10T18:53:37.434336+0000","last_became_active":"2024-09-10T18:53:30.318898+0000","last_became_peered":"2024-09-10T18:53:30.318898+0000","last_unstale":"2024-09-10T18:53:37.434336+0000","last_undegraded":"2024-09-10T18:53:37.434336+0000","last_fullsized":"2024-09-10T18:53:37.434336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:06.510757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151462+0000","last_change":"2024-09-10T18:53:26.437618+0000","last_active":"2024-09-10T18:53:37.151462+0000","last_peered":"2024-09-10T18:53:37.151462+0000","last_clean":"2024-09-10T18:53:37.151462+0000","last_became_active":"2024-09-10T18:53:26.437316+0000","last_became_peered":"2024-09-10T18:53:26.437316+0000","last_unstale":"2024-09-10T18:53:37.151462+0000","last_undegraded":"2024-09-10T18:53:37.151462+0000","last_fullsized":"2024-09-10T18:53:37.151462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:17.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434442+0000","last_change":"2024-09-10T18:53:24.289383+0000","last_active":"2024-09-10T18:53:37.434442+0000","last_peered":"2024-09-10T18:53:37.434442+0000","last_clean":"2024-09-10T18:53:37.434442+0000","last_became_active":"2024-09-10T18:53:24.289122+0000","last_became_peered":"2024-09-10T18:53:24.289122+0000","last_unstale":"2024-09-10T18:53:37.434442+0000","last_undegraded":"2024-09-10T18:53:37.434442+0000","last_fullsized":"2024-09-10T18:53:37.434442+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:59.862994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435546+0000","last_change":"2024-09-10T18:53:28.304502+0000","last_active":"2024-09-10T18:53:37.435546+0000","last_peered":"2024-09-10T18:53:37.435546+0000","last_clean":"2024-09-10T18:53:37.435546+0000","last_became_active":"2024-09-10T18:53:28.304254+0000","last_became_peered":"2024-09-10T18:53:28.304254+0000","last_unstale":"2024-09-10T18:53:37.435546+0000","last_undegraded":"2024-09-10T18:53:37.435546+0000","last_fullsized":"2024-09-10T18:53:37.435546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:21.344115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435095+0000","last_change":"2024-09-10T18:53:30.317564+0000","last_active":"2024-09-10T18:53:37.435095+0000","last_peered":"2024-09-10T18:53:37.435095+0000","last_clean":"2024-09-10T18:53:37.435095+0000","last_became_active":"2024-09-10T18:53:30.317271+0000","last_became_peered":"2024-09-10T18:53:30.317271+0000","last_unstale":"2024-09-10T18:53:37.435095+0000","last_undegraded":"2024-09-10T18:53:37.435095+0000","last_fullsized":"2024-09-10T18:53:37.435095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:18.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363121+0000","last_change":"2024-09-10T18:53:24.315004+0000","last_active":"2024-09-10T18:53:37.363121+0000","last_peered":"2024-09-10T18:53:37.363121+0000","last_clean":"2024-09-10T18:53:37.363121+0000","last_became_active":"2024-09-10T18:53:24.314611+0000","last_became_peered":"2024-09-10T18:53:24.314611+0000","last_unstale":"2024-09-10T18:53:37.363121+0000","last_undegraded":"2024-09-10T18:53:37.363121+0000","last_fullsized":"2024-09-10T18:53:37.363121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:43.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435570+0000","last_change":"2024-09-10T18:53:26.436607+0000","last_active":"2024-09-10T18:53:37.435570+0000","last_peered":"2024-09-10T18:53:37.435570+0000","last_clean":"2024-09-10T18:53:37.435570+0000","last_became_active":"2024-09-10T18:53:26.436341+0000","last_became_peered":"2024-09-10T18:53:26.436341+0000","last_unstale":"2024-09-10T18:53:37.435570+0000","last_undegraded":"2024-09-10T18:53:37.435570+0000","last_fullsized":"2024-09-10T18:53:37.435570+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:17.058998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435299+0000","last_change":"2024-09-10T18:53:28.303261+0000","last_active":"2024-09-10T18:53:37.435299+0000","last_peered":"2024-09-10T18:53:37.435299+0000","last_clean":"2024-09-10T18:53:37.435299+0000","last_became_active":"2024-09-10T18:53:28.302954+0000","last_became_peered":"2024-09-10T18:53:28.302954+0000","last_unstale":"2024-09-10T18:53:37.435299+0000","last_undegraded":"2024-09-10T18:53:37.435299+0000","last_fullsized":"2024-09-10T18:53:37.435299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:16.935788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434988+0000","last_change":"2024-09-10T18:53:30.494848+0000","last_active":"2024-09-10T18:53:37.434988+0000","last_peered":"2024-09-10T18:53:37.434988+0000","last_clean":"2024-09-10T18:53:37.434988+0000","last_became_active":"2024-09-10T18:53:30.494590+0000","last_became_peered":"2024-09-10T18:53:30.494590+0000","last_unstale":"2024-09-10T18:53:37.434988+0000","last_undegraded":"2024-09-10T18:53:37.434988+0000","last_fullsized":"2024-09-10T18:53:37.434988+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:16.904105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435118+0000","last_change":"2024-09-10T18:53:26.434347+0000","last_active":"2024-09-10T18:53:37.435118+0000","last_peered":"2024-09-10T18:53:37.435118+0000","last_clean":"2024-09-10T18:53:37.435118+0000","last_became_active":"2024-09-10T18:53:26.434069+0000","last_became_peered":"2024-09-10T18:53:26.434069+0000","last_unstale":"2024-09-10T18:53:37.435118+0000","last_undegraded":"2024-09-10T18:53:37.435118+0000","last_fullsized":"2024-09-10T18:53:37.435118+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:08.159776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434802+0000","last_change":"2024-09-10T18:53:24.315075+0000","last_active":"2024-09-10T18:53:37.434802+0000","last_peered":"2024-09-10T18:53:37.434802+0000","last_clean":"2024-09-10T18:53:37.434802+0000","last_became_active":"2024-09-10T18:53:24.314551+0000","last_became_peered":"2024-09-10T18:53:24.314551+0000","last_unstale":"2024-09-10T18:53:37.434802+0000","last_undegraded":"2024-09-10T18:53:37.434802+0000","last_fullsized":"2024-09-10T18:53:37.434802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.067240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543099+0000","last_change":"2024-09-10T18:53:28.302415+0000","last_active":"2024-09-10T18:54:37.543099+0000","last_peered":"2024-09-10T18:54:37.543099+0000","last_clean":"2024-09-10T18:54:37.543099+0000","last_became_active":"2024-09-10T18:53:28.302125+0000","last_became_peered":"2024-09-10T18:53:28.302125+0000","last_unstale":"2024-09-10T18:54:37.543099+0000","last_undegraded":"2024-09-10T18:54:37.543099+0000","last_fullsized":"2024-09-10T18:54:37.543099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:51.115976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151073+0000","last_change":"2024-09-10T18:53:30.318114+0000","last_active":"2024-09-10T18:53:37.151073+0000","last_peered":"2024-09-10T18:53:37.151073+0000","last_clean":"2024-09-10T18:53:37.151073+0000","last_became_active":"2024-09-10T18:53:30.317816+0000","last_became_peered":"2024-09-10T18:53:30.317816+0000","last_unstale":"2024-09-10T18:53:37.151073+0000","last_undegraded":"2024-09-10T18:53:37.151073+0000","last_fullsized":"2024-09-10T18:53:37.151073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:55.124939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434564+0000","last_change":"2024-09-10T18:53:26.434624+0000","last_active":"2024-09-10T18:53:37.434564+0000","last_peered":"2024-09-10T18:53:37.434564+0000","last_clean":"2024-09-10T18:53:37.434564+0000","last_became_active":"2024-09-10T18:53:26.434286+0000","last_became_peered":"2024-09-10T18:53:26.434286+0000","last_unstale":"2024-09-10T18:53:37.434564+0000","last_undegraded":"2024-09-10T18:53:37.434564+0000","last_fullsized":"2024-09-10T18:53:37.434564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:17.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151511+0000","last_change":"2024-09-10T18:53:24.289506+0000","last_active":"2024-09-10T18:53:37.151511+0000","last_peered":"2024-09-10T18:53:37.151511+0000","last_clean":"2024-09-10T18:53:37.151511+0000","last_became_active":"2024-09-10T18:53:24.289093+0000","last_became_peered":"2024-09-10T18:53:24.289093+0000","last_unstale":"2024-09-10T18:53:37.151511+0000","last_undegraded":"2024-09-10T18:53:37.151511+0000","last_fullsized":"2024-09-10T18:53:37.151511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:53.254037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542752+0000","last_change":"2024-09-10T18:53:28.306676+0000","last_active":"2024-09-10T18:54:37.542752+0000","last_peered":"2024-09-10T18:54:37.542752+0000","last_clean":"2024-09-10T18:54:37.542752+0000","last_became_active":"2024-09-10T18:53:28.306371+0000","last_became_peered":"2024-09-10T18:53:28.306371+0000","last_unstale":"2024-09-10T18:54:37.542752+0000","last_undegraded":"2024-09-10T18:54:37.542752+0000","last_fullsized":"2024-09-10T18:54:37.542752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:16.511464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148762+0000","last_change":"2024-09-10T18:53:30.319982+0000","last_active":"2024-09-10T18:53:37.148762+0000","last_peered":"2024-09-10T18:53:37.148762+0000","last_clean":"2024-09-10T18:53:37.148762+0000","last_became_active":"2024-09-10T18:53:30.319386+0000","last_became_peered":"2024-09-10T18:53:30.319386+0000","last_unstale":"2024-09-10T18:53:37.148762+0000","last_undegraded":"2024-09-10T18:53:37.148762+0000","last_fullsized":"2024-09-10T18:53:37.148762+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:42.892780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150492+0000","last_change":"2024-09-10T18:53:26.301906+0000","last_active":"2024-09-10T18:53:37.150492+0000","last_peered":"2024-09-10T18:53:37.150492+0000","last_clean":"2024-09-10T18:53:37.150492+0000","last_became_active":"2024-09-10T18:53:26.301613+0000","last_became_peered":"2024-09-10T18:53:26.301613+0000","last_unstale":"2024-09-10T18:53:37.150492+0000","last_undegraded":"2024-09-10T18:53:37.150492+0000","last_fullsized":"2024-09-10T18:53:37.150492+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:28.086658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363259+0000","last_change":"2024-09-10T18:53:24.317867+0000","last_active":"2024-09-10T18:53:37.363259+0000","last_peered":"2024-09-10T18:53:37.363259+0000","last_clean":"2024-09-10T18:53:37.363259+0000","last_became_active":"2024-09-10T18:53:24.317382+0000","last_became_peered":"2024-09-10T18:53:24.317382+0000","last_unstale":"2024-09-10T18:53:37.363259+0000","last_undegraded":"2024-09-10T18:53:37.363259+0000","last_fullsized":"2024-09-10T18:53:37.363259+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:12.670061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434557+0000","last_change":"2024-09-10T18:53:28.303638+0000","last_active":"2024-09-10T18:53:37.434557+0000","last_peered":"2024-09-10T18:53:37.434557+0000","last_clean":"2024-09-10T18:53:37.434557+0000","last_became_active":"2024-09-10T18:53:28.303301+0000","last_became_peered":"2024-09-10T18:53:28.303301+0000","last_unstale":"2024-09-10T18:53:37.434557+0000","last_undegraded":"2024-09-10T18:53:37.434557+0000","last_fullsized":"2024-09-10T18:53:37.434557+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:54.744610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150732+0000","last_change":"2024-09-10T18:53:30.321476+0000","last_active":"2024-09-10T18:53:37.150732+0000","last_peered":"2024-09-10T18:53:37.150732+0000","last_clean":"2024-09-10T18:53:37.150732+0000","last_became_active":"2024-09-10T18:53:30.321183+0000","last_became_peered":"2024-09-10T18:53:30.321183+0000","last_unstale":"2024-09-10T18:53:37.150732+0000","last_undegraded":"2024-09-10T18:53:37.150732+0000","last_fullsized":"2024-09-10T18:53:37.150732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.580166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148276+0000","last_change":"2024-09-10T18:53:26.297537+0000","last_active":"2024-09-10T18:53:37.148276+0000","last_peered":"2024-09-10T18:53:37.148276+0000","last_clean":"2024-09-10T18:53:37.148276+0000","last_became_active":"2024-09-10T18:53:26.297170+0000","last_became_peered":"2024-09-10T18:53:26.297170+0000","last_unstale":"2024-09-10T18:53:37.148276+0000","last_undegraded":"2024-09-10T18:53:37.148276+0000","last_fullsized":"2024-09-10T18:53:37.148276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:42.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150489+0000","last_change":"2024-09-10T18:53:24.311844+0000","last_active":"2024-09-10T18:53:37.150489+0000","last_peered":"2024-09-10T18:53:37.150489+0000","last_clean":"2024-09-10T18:53:37.150489+0000","last_became_active":"2024-09-10T18:53:24.311613+0000","last_became_peered":"2024-09-10T18:53:24.311613+0000","last_unstale":"2024-09-10T18:53:37.150489+0000","last_undegraded":"2024-09-10T18:53:37.150489+0000","last_fullsized":"2024-09-10T18:53:37.150489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:18.967074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150402+0000","last_change":"2024-09-10T18:53:28.310289+0000","last_active":"2024-09-10T18:53:37.150402+0000","last_peered":"2024-09-10T18:53:37.150402+0000","last_clean":"2024-09-10T18:53:37.150402+0000","last_became_active":"2024-09-10T18:53:28.310005+0000","last_became_peered":"2024-09-10T18:53:28.310005+0000","last_unstale":"2024-09-10T18:53:37.150402+0000","last_undegraded":"2024-09-10T18:53:37.150402+0000","last_fullsized":"2024-09-10T18:53:37.150402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:54.223793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434162+0000","last_change":"2024-09-10T18:53:30.315912+0000","last_active":"2024-09-10T18:53:37.434162+0000","last_peered":"2024-09-10T18:53:37.434162+0000","last_clean":"2024-09-10T18:53:37.434162+0000","last_became_active":"2024-09-10T18:53:30.315634+0000","last_became_peered":"2024-09-10T18:53:30.315634+0000","last_unstale":"2024-09-10T18:53:37.434162+0000","last_undegraded":"2024-09-10T18:53:37.434162+0000","last_fullsized":"2024-09-10T18:53:37.434162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:45.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434172+0000","last_change":"2024-09-10T18:53:26.297801+0000","last_active":"2024-09-10T18:53:37.434172+0000","last_peered":"2024-09-10T18:53:37.434172+0000","last_clean":"2024-09-10T18:53:37.434172+0000","last_became_active":"2024-09-10T18:53:26.297441+0000","last_became_peered":"2024-09-10T18:53:26.297441+0000","last_unstale":"2024-09-10T18:53:37.434172+0000","last_undegraded":"2024-09-10T18:53:37.434172+0000","last_fullsized":"2024-09-10T18:53:37.434172+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:49.951831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435414+0000","last_change":"2024-09-10T18:53:24.315871+0000","last_active":"2024-09-10T18:53:37.435414+0000","last_peered":"2024-09-10T18:53:37.435414+0000","last_clean":"2024-09-10T18:53:37.435414+0000","last_became_active":"2024-09-10T18:53:24.315514+0000","last_became_peered":"2024-09-10T18:53:24.315514+0000","last_unstale":"2024-09-10T18:53:37.435414+0000","last_undegraded":"2024-09-10T18:53:37.435414+0000","last_fullsized":"2024-09-10T18:53:37.435414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:05.010080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.362938+0000","last_change":"2024-09-10T18:53:28.300959+0000","last_active":"2024-09-10T18:53:37.362938+0000","last_peered":"2024-09-10T18:53:37.362938+0000","last_clean":"2024-09-10T18:53:37.362938+0000","last_became_active":"2024-09-10T18:53:28.300657+0000","last_became_peered":"2024-09-10T18:53:28.300657+0000","last_unstale":"2024-09-10T18:53:37.362938+0000","last_undegraded":"2024-09-10T18:53:37.362938+0000","last_fullsized":"2024-09-10T18:53:37.362938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:48.938593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434702+0000","last_change":"2024-09-10T18:53:30.318413+0000","last_active":"2024-09-10T18:53:37.434702+0000","last_peered":"2024-09-10T18:53:37.434702+0000","last_clean":"2024-09-10T18:53:37.434702+0000","last_became_active":"2024-09-10T18:53:30.318138+0000","last_became_peered":"2024-09-10T18:53:30.318138+0000","last_unstale":"2024-09-10T18:53:37.434702+0000","last_undegraded":"2024-09-10T18:53:37.434702+0000","last_fullsized":"2024-09-10T18:53:37.434702+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:16.909131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434249+0000","last_change":"2024-09-10T18:53:26.434338+0000","last_active":"2024-09-10T18:53:37.434249+0000","last_peered":"2024-09-10T18:53:37.434249+0000","last_clean":"2024-09-10T18:53:37.434249+0000","last_became_active":"2024-09-10T18:53:26.434083+0000","last_became_peered":"2024-09-10T18:53:26.434083+0000","last_unstale":"2024-09-10T18:53:37.434249+0000","last_undegraded":"2024-09-10T18:53:37.434249+0000","last_fullsized":"2024-09-10T18:53:37.434249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:10.922765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363510+0000","last_change":"2024-09-10T18:53:24.317730+0000","last_active":"2024-09-10T18:53:37.363510+0000","last_peered":"2024-09-10T18:53:37.363510+0000","last_clean":"2024-09-10T18:53:37.363510+0000","last_became_active":"2024-09-10T18:53:24.317363+0000","last_became_peered":"2024-09-10T18:53:24.317363+0000","last_unstale":"2024-09-10T18:53:37.363510+0000","last_undegraded":"2024-09-10T18:53:37.363510+0000","last_fullsized":"2024-09-10T18:53:37.363510+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.286267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434251+0000","last_change":"2024-09-10T18:53:28.299121+0000","last_active":"2024-09-10T18:53:37.434251+0000","last_peered":"2024-09-10T18:53:37.434251+0000","last_clean":"2024-09-10T18:53:37.434251+0000","last_became_active":"2024-09-10T18:53:28.298766+0000","last_became_peered":"2024-09-10T18:53:28.298766+0000","last_unstale":"2024-09-10T18:53:37.434251+0000","last_undegraded":"2024-09-10T18:53:37.434251+0000","last_fullsized":"2024-09-10T18:53:37.434251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:47.151320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434246+0000","last_change":"2024-09-10T18:53:30.495927+0000","last_active":"2024-09-10T18:53:37.434246+0000","last_peered":"2024-09-10T18:53:37.434246+0000","last_clean":"2024-09-10T18:53:37.434246+0000","last_became_active":"2024-09-10T18:53:30.495510+0000","last_became_peered":"2024-09-10T18:53:30.495510+0000","last_unstale":"2024-09-10T18:53:37.434246+0000","last_undegraded":"2024-09-10T18:53:37.434246+0000","last_fullsized":"2024-09-10T18:53:37.434246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:42.354276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435463+0000","last_change":"2024-09-10T18:53:30.318875+0000","last_active":"2024-09-10T18:53:37.435463+0000","last_peered":"2024-09-10T18:53:37.435463+0000","last_clean":"2024-09-10T18:53:37.435463+0000","last_became_active":"2024-09-10T18:53:30.318611+0000","last_became_peered":"2024-09-10T18:53:30.318611+0000","last_unstale":"2024-09-10T18:53:37.435463+0000","last_undegraded":"2024-09-10T18:53:37.435463+0000","last_fullsized":"2024-09-10T18:53:37.435463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:22.747007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434632+0000","last_change":"2024-09-10T18:53:24.317163+0000","last_active":"2024-09-10T18:53:37.434632+0000","last_peered":"2024-09-10T18:53:37.434632+0000","last_clean":"2024-09-10T18:53:37.434632+0000","last_became_active":"2024-09-10T18:53:24.315764+0000","last_became_peered":"2024-09-10T18:53:24.315764+0000","last_unstale":"2024-09-10T18:53:37.434632+0000","last_undegraded":"2024-09-10T18:53:37.434632+0000","last_fullsized":"2024-09-10T18:53:37.434632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:14.759149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151328+0000","last_change":"2024-09-10T18:53:26.437213+0000","last_active":"2024-09-10T18:53:37.151328+0000","last_peered":"2024-09-10T18:53:37.151328+0000","last_clean":"2024-09-10T18:53:37.151328+0000","last_became_active":"2024-09-10T18:53:26.436935+0000","last_became_peered":"2024-09-10T18:53:26.436935+0000","last_unstale":"2024-09-10T18:53:37.151328+0000","last_undegraded":"2024-09-10T18:53:37.151328+0000","last_fullsized":"2024-09-10T18:53:37.151328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:04.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":43,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:32.541603+0000","last_change":"2024-09-10T18:53:28.301965+0000","last_active":"2024-09-10T18:54:32.541603+0000","last_peered":"2024-09-10T18:54:32.541603+0000","last_clean":"2024-09-10T18:54:32.541603+0000","last_became_active":"2024-09-10T18:53:28.301640+0000","last_became_peered":"2024-09-10T18:53:28.301640+0000","last_unstale":"2024-09-10T18:54:32.541603+0000","last_undegraded":"2024-09-10T18:54:32.541603+0000","last_fullsized":"2024-09-10T18:54:32.541603+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.876161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150025+0000","last_change":"2024-09-10T18:53:30.317510+0000","last_active":"2024-09-10T18:53:37.150025+0000","last_peered":"2024-09-10T18:53:37.150025+0000","last_clean":"2024-09-10T18:53:37.150025+0000","last_became_active":"2024-09-10T18:53:30.317222+0000","last_became_peered":"2024-09-10T18:53:30.317222+0000","last_unstale":"2024-09-10T18:53:37.150025+0000","last_undegraded":"2024-09-10T18:53:37.150025+0000","last_fullsized":"2024-09-10T18:53:37.150025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:31.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434453+0000","last_change":"2024-09-10T18:53:24.315766+0000","last_active":"2024-09-10T18:53:37.434453+0000","last_peered":"2024-09-10T18:53:37.434453+0000","last_clean":"2024-09-10T18:53:37.434453+0000","last_became_active":"2024-09-10T18:53:24.315411+0000","last_became_peered":"2024-09-10T18:53:24.315411+0000","last_unstale":"2024-09-10T18:53:37.434453+0000","last_undegraded":"2024-09-10T18:53:37.434453+0000","last_fullsized":"2024-09-10T18:53:37.434453+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:02.358653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363480+0000","last_change":"2024-09-10T18:53:26.299889+0000","last_active":"2024-09-10T18:53:37.363480+0000","last_peered":"2024-09-10T18:53:37.363480+0000","last_clean":"2024-09-10T18:53:37.363480+0000","last_became_active":"2024-09-10T18:53:26.299584+0000","last_became_peered":"2024-09-10T18:53:26.299584+0000","last_unstale":"2024-09-10T18:53:37.363480+0000","last_undegraded":"2024-09-10T18:53:37.363480+0000","last_fullsized":"2024-09-10T18:53:37.363480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:39.636331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151236+0000","last_change":"2024-09-10T18:53:28.309275+0000","last_active":"2024-09-10T18:53:37.151236+0000","last_peered":"2024-09-10T18:53:37.151236+0000","last_clean":"2024-09-10T18:53:37.151236+0000","last_became_active":"2024-09-10T18:53:28.308803+0000","last_became_peered":"2024-09-10T18:53:28.308803+0000","last_unstale":"2024-09-10T18:53:37.151236+0000","last_undegraded":"2024-09-10T18:53:37.151236+0000","last_fullsized":"2024-09-10T18:53:37.151236+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:50.786087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148652+0000","last_change":"2024-09-10T18:53:30.493741+0000","last_active":"2024-09-10T18:53:37.148652+0000","last_peered":"2024-09-10T18:53:37.148652+0000","last_clean":"2024-09-10T18:53:37.148652+0000","last_became_active":"2024-09-10T18:53:30.493445+0000","last_became_peered":"2024-09-10T18:53:30.493445+0000","last_unstale":"2024-09-10T18:53:37.148652+0000","last_undegraded":"2024-09-10T18:53:37.148652+0000","last_fullsized":"2024-09-10T18:53:37.148652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:02.419396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363394+0000","last_change":"2024-09-10T18:53:24.316691+0000","last_active":"2024-09-10T18:53:37.363394+0000","last_peered":"2024-09-10T18:53:37.363394+0000","last_clean":"2024-09-10T18:53:37.363394+0000","last_became_active":"2024-09-10T18:53:24.316415+0000","last_became_peered":"2024-09-10T18:53:24.316415+0000","last_unstale":"2024-09-10T18:53:37.363394+0000","last_undegraded":"2024-09-10T18:53:37.363394+0000","last_fullsized":"2024-09-10T18:53:37.363394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:21.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435351+0000","last_change":"2024-09-10T18:53:26.303843+0000","last_active":"2024-09-10T18:53:37.435351+0000","last_peered":"2024-09-10T18:53:37.435351+0000","last_clean":"2024-09-10T18:53:37.435351+0000","last_became_active":"2024-09-10T18:53:26.303472+0000","last_became_peered":"2024-09-10T18:53:26.303472+0000","last_unstale":"2024-09-10T18:53:37.435351+0000","last_undegraded":"2024-09-10T18:53:37.435351+0000","last_fullsized":"2024-09-10T18:53:37.435351+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:00.330500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150180+0000","last_change":"2024-09-10T18:53:28.302606+0000","last_active":"2024-09-10T18:53:37.150180+0000","last_peered":"2024-09-10T18:53:37.150180+0000","last_clean":"2024-09-10T18:53:37.150180+0000","last_became_active":"2024-09-10T18:53:28.302313+0000","last_became_peered":"2024-09-10T18:53:28.302313+0000","last_unstale":"2024-09-10T18:53:37.150180+0000","last_undegraded":"2024-09-10T18:53:37.150180+0000","last_fullsized":"2024-09-10T18:53:37.150180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:21.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435483+0000","last_change":"2024-09-10T18:53:30.493777+0000","last_active":"2024-09-10T18:53:37.435483+0000","last_peered":"2024-09-10T18:53:37.435483+0000","last_clean":"2024-09-10T18:53:37.435483+0000","last_became_active":"2024-09-10T18:53:30.493491+0000","last_became_peered":"2024-09-10T18:53:30.493491+0000","last_unstale":"2024-09-10T18:53:37.435483+0000","last_undegraded":"2024-09-10T18:53:37.435483+0000","last_fullsized":"2024-09-10T18:53:37.435483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:54.114679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434649+0000","last_change":"2024-09-10T18:53:24.321515+0000","last_active":"2024-09-10T18:53:37.434649+0000","last_peered":"2024-09-10T18:53:37.434649+0000","last_clean":"2024-09-10T18:53:37.434649+0000","last_became_active":"2024-09-10T18:53:24.321197+0000","last_became_peered":"2024-09-10T18:53:24.321197+0000","last_unstale":"2024-09-10T18:53:37.434649+0000","last_undegraded":"2024-09-10T18:53:37.434649+0000","last_fullsized":"2024-09-10T18:53:37.434649+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:56.925730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435390+0000","last_change":"2024-09-10T18:53:26.291345+0000","last_active":"2024-09-10T18:53:37.435390+0000","last_peered":"2024-09-10T18:53:37.435390+0000","last_clean":"2024-09-10T18:53:37.435390+0000","last_became_active":"2024-09-10T18:53:26.291067+0000","last_became_peered":"2024-09-10T18:53:26.291067+0000","last_unstale":"2024-09-10T18:53:37.435390+0000","last_undegraded":"2024-09-10T18:53:37.435390+0000","last_fullsized":"2024-09-10T18:53:37.435390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:28.293789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148492+0000","last_change":"2024-09-10T18:53:28.306652+0000","last_active":"2024-09-10T18:53:37.148492+0000","last_peered":"2024-09-10T18:53:37.148492+0000","last_clean":"2024-09-10T18:53:37.148492+0000","last_became_active":"2024-09-10T18:53:28.306329+0000","last_became_peered":"2024-09-10T18:53:28.306329+0000","last_unstale":"2024-09-10T18:53:37.148492+0000","last_undegraded":"2024-09-10T18:53:37.148492+0000","last_fullsized":"2024-09-10T18:53:37.148492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1269760,"data_stored":868553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42599999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.40799999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59899999999999998}]}]},{"osd":6,"up_from":42,"seq":180388626457,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298572,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446580,"statfs":{"total":95995035648,"available":95689297920,"internally_reserved":0,"allocated":1249280,"data_stored":867138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.61899999999999999}]},{"osd":1,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.58399999999999996}]},{"osd":2,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.51700000000000002}]},{"osd":3,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.53200000000000003}]},{"osd":4,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.65400000000000003}]},{"osd":5,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.54900000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.67200000000000004}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":663552,"data_stored":276987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65400000000000003,"5min":1.387,"15min":1.387},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69199999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71699999999999997,"5min":1.496,"15min":1.496},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.42799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.69299999999999995,"5min":1.534,"15min":1.534},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.443,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":1.427,"15min":1.427},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74099999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.32400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71099999999999997}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":277524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.68200000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53200000000000003,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.71999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.872,"5min":1.052,"15min":1.052},"last":0.432},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63800000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.498}]},{"osd":2,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.443}]},{"osd":3,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67500000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65700000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:54:26 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.36599999999999999,"15min":0.35499999999999998},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.35799999999999998},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34999999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.3010000000000002,"5min":7.3010000000000002,"15min":7.3010000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":0.40400000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.5449999999999999,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":0.38400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298040,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447112,"statfs":{"total":95995035648,"available":95689842688,"internally_reserved":0,"allocated":700416,"data_stored":278400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.374,"15min":0.38100000000000001},"min":{"1min":0.20300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.501,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.44}]},{"osd":1,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40300000000000002,"15min":0.41899999999999998},"min":{"1min":0.255,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.42699999999999999,"15min":0.434},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.379},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.41499999999999998,"15min":0.434},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.42399999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85399999999999998,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81200000000000006,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:54:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":1.238,"15min":1.238},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Sep 10 18:53:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":277152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36299999999999999,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.376,"15min":0.35699999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.36699999999999999},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42599999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.378}]},{"osd":4,"last update":"Tue Sep 10 18:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.754,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.495}]},{"osd":6,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:54:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298060,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447092,"statfs":{"total":95995035648,"available":95689822208,"internally_reserved":0,"allocated":720896,"data_stored":279572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.39900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.70699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36299999999999999,"15min":0.35799999999999998},"min":{"1min":0.248,"5min":0.215,"15min":0.215},"max":{"1min":0.52600000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39300000000000002,"15min":0.40000000000000002},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.48699999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.39900000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.623}]},{"osd":5,"last update":"Tue Sep 10 18:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72399999999999998,"5min":1.417,"15min":1.417},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1265664,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:53:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.29399999999999998},"min":{"1min":0.26300000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.52200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.33100000000000002,"15min":0.29899999999999999},"min":{"1min":0.22700000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:54:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40000000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.252}]},{"osd":3,"last update":"Tue Sep 10 18:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44900000000000001,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.82299999999999995,"15min":0.872},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.73499999999999999,"5min":7.8159999999999998,"15min":7.8159999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.871,"15min":0.92200000000000004},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.433}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:54:49.818 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph pg dump --format=json 2024-09-10T18:54:50.065 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:50.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:50.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[40355]: Deploying daemon grafana.a on smithi162 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:50 smithi107 ceph-mon[29404]: Deploying daemon grafana.a on smithi162 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: pgmap v27: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-ssl-verify", "value": "false"}]: dispatch 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-mon[33228]: Deploying daemon grafana.a on smithi162 2024-09-10T18:54:51.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:54:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:54:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:54:51.215 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:51 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:51.019Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000397339s 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 -- 172.21.15.107:0/3777420358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 msgr2=0x7fca7c0fb220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 --2- 172.21.15.107:0/3777420358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fb220 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fca64004890 tx=0x7fca64009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 -- 172.21.15.107:0/3777420358 shutdown_connections 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 --2- 172.21.15.107:0/3777420358 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c109690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 --2- 172.21.15.107:0/3777420358 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fb220 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.814 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 --2- 172.21.15.107:0/3777420358 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 0x7fca7c102cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 -- 172.21.15.107:0/3777420358 >> 172.21.15.107:0/3777420358 conn(0x7fca7c074d40 msgr2=0x7fca7c0f9140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 -- 172.21.15.107:0/3777420358 shutdown_connections 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.794+0000 7fca810d7640 1 -- 172.21.15.107:0/3777420358 wait complete. 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 Processor -- start 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 -- start start 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c0fc4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 0x7fca7c0fca10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c11b2b0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c11b420 con 0x7fca7c102900 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca810d7640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca7c11b5e0 con 0x7fca7c0fb760 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:51.815 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca73fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c0fc4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48926/0 (socket says 172.21.15.107:48926) 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 -- 172.21.15.107:0/3318030401 learned_addr learned my addr 172.21.15.107:0/3318030401 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7b59e640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 0x7fca7c0fca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 -- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 msgr2=0x7fca7c0fc4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c0fc4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 -- 172.21.15.107:0/3318030401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 msgr2=0x7fca7c0fca10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 0x7fca7c0fca10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 -- 172.21.15.107:0/3318030401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca740098b0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca73fff640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c0fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.795+0000 7fca7ad9d640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ebf0 tx=0x7fca6c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.796+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c007c30 con 0x7fca7c0fadb0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.796+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca64005040 con 0x7fca7c0fadb0 2024-09-10T18:54:51.816 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.796+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fca6c00f040 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.796+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c0093e0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.796+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca7c11bbc0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.797+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca7c0fe0d0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.797+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fca6c009750 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.798+0000 7fca737fe640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 0x7fca680713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.798+0000 7fca73fff640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 0x7fca680713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.798+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7fca6c08e670 con 0x7fca7c0fadb0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.798+0000 7fca73fff640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 0x7fca680713c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca64009040 tx=0x7fca64009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:51.817 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.802+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fca6c04e2a0 con 0x7fca7c0fadb0 2024-09-10T18:54:51.956 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.954+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 --> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fca7c10bbf0 con 0x7fca6806ef10 2024-09-10T18:54:51.967 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.966+0000 7fca737fe640 1 -- 172.21.15.107:0/3318030401 <== mgr.14559 v2:172.21.15.107:6800/339792131 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+383537 (secure 0 0 0) 0x7fca7c10bbf0 con 0x7fca6806ef10 2024-09-10T18:54:51.971 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:54:51.973 INFO:teuthology.orchestra.run.smithi107.stderr:dumped all 2024-09-10T18:54:51.975 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 msgr2=0x7fca680713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 0x7fca680713c0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca64009040 tx=0x7fca64009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 msgr2=0x7fca7c0fbf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00ebf0 tx=0x7fca6c00b040 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 shutdown_connections 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fca6806ef10 0x7fca680713c0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fca7c102900 0x7fca7c0fca10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fca7c0fb760 0x7fca7c0fc4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 --2- 172.21.15.107:0/3318030401 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fca7c0fadb0 0x7fca7c0fbf90 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.969+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 >> 172.21.15.107:0/3318030401 conn(0x7fca7c074d40 msgr2=0x7fca7c072030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.970+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 shutdown_connections 2024-09-10T18:54:51.976 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:51.970+0000 7fca810d7640 1 -- 172.21.15.107:0/3318030401 wait complete. 2024-09-10T18:54:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:52 smithi107 ceph-mon[29404]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:52 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:52.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:52 smithi107 ceph-mon[40355]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:52.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:52 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:52 smithi162 ceph-mon[33228]: pgmap v28: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:52 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:54:52.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:54:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:54:52.754 INFO:teuthology.orchestra.run.smithi107.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2024-09-10T18:54:51.040752+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596188,"num_objects":196,"num_object_clones":0,"num_object_copies":588,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":196,"num_whiteouts":0,"num_read":927,"num_read_kb":784,"num_write":543,"num_write_kb":1671,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":548,"ondisk_log_size":548,"up":396,"acting":396,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":396,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":749961216,"kb_used":2385876,"kb_used_data":7036,"kb_used_omap":13,"kb_used_meta":2378738,"kb_avail":747575340,"statfs":{"total":767960285184,"available":765517148160,"internally_reserved":0,"allocated":7204864,"data_stored":3992702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":13632,"internal_metadata":2435828416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002761"},"pg_stats":[{"pgid":"6.1b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434474+0000","last_change":"2024-09-10T18:53:30.496366+0000","last_active":"2024-09-10T18:53:37.434474+0000","last_peered":"2024-09-10T18:53:37.434474+0000","last_clean":"2024-09-10T18:53:37.434474+0000","last_became_active":"2024-09-10T18:53:30.495541+0000","last_became_peered":"2024-09-10T18:53:30.495541+0000","last_unstale":"2024-09-10T18:53:37.434474+0000","last_undegraded":"2024-09-10T18:53:37.434474+0000","last_fullsized":"2024-09-10T18:53:37.434474+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:59:09.551565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151354+0000","last_change":"2024-09-10T18:53:24.321406+0000","last_active":"2024-09-10T18:53:37.151354+0000","last_peered":"2024-09-10T18:53:37.151354+0000","last_clean":"2024-09-10T18:53:37.151354+0000","last_became_active":"2024-09-10T18:53:24.321125+0000","last_became_peered":"2024-09-10T18:53:24.321125+0000","last_unstale":"2024-09-10T18:53:37.151354+0000","last_undegraded":"2024-09-10T18:53:37.151354+0000","last_fullsized":"2024-09-10T18:53:37.151354+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:28:46.179922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,4],"acting":[0,7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.19","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434324+0000","last_change":"2024-09-10T18:53:26.298914+0000","last_active":"2024-09-10T18:53:37.434324+0000","last_peered":"2024-09-10T18:53:37.434324+0000","last_clean":"2024-09-10T18:53:37.434324+0000","last_became_active":"2024-09-10T18:53:26.298589+0000","last_became_peered":"2024-09-10T18:53:26.298589+0000","last_unstale":"2024-09-10T18:53:37.434324+0000","last_undegraded":"2024-09-10T18:53:37.434324+0000","last_fullsized":"2024-09-10T18:53:37.434324+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:58:06.627755+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,0],"acting":[3,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.147977+0000","last_change":"2024-09-10T18:53:28.305555+0000","last_active":"2024-09-10T18:53:37.147977+0000","last_peered":"2024-09-10T18:53:37.147977+0000","last_clean":"2024-09-10T18:53:37.147977+0000","last_became_active":"2024-09-10T18:53:28.305184+0000","last_became_peered":"2024-09-10T18:53:28.305184+0000","last_unstale":"2024-09-10T18:53:37.147977+0000","last_undegraded":"2024-09-10T18:53:37.147977+0000","last_fullsized":"2024-09-10T18:53:37.147977+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:59:50.501427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435255+0000","last_change":"2024-09-10T18:53:24.312553+0000","last_active":"2024-09-10T18:53:37.435255+0000","last_peered":"2024-09-10T18:53:37.435255+0000","last_clean":"2024-09-10T18:53:37.435255+0000","last_became_active":"2024-09-10T18:53:24.312219+0000","last_became_peered":"2024-09-10T18:53:24.312219+0000","last_unstale":"2024-09-10T18:53:37.435255+0000","last_undegraded":"2024-09-10T18:53:37.435255+0000","last_fullsized":"2024-09-10T18:53:37.435255+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:57:35.218940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.147993+0000","last_change":"2024-09-10T18:53:26.298649+0000","last_active":"2024-09-10T18:53:37.147993+0000","last_peered":"2024-09-10T18:53:37.147993+0000","last_clean":"2024-09-10T18:53:37.147993+0000","last_became_active":"2024-09-10T18:53:26.298353+0000","last_became_peered":"2024-09-10T18:53:26.298353+0000","last_unstale":"2024-09-10T18:53:37.147993+0000","last_undegraded":"2024-09-10T18:53:37.147993+0000","last_fullsized":"2024-09-10T18:53:37.147993+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:31:23.749365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149705+0000","last_change":"2024-09-10T18:53:28.299218+0000","last_active":"2024-09-10T18:53:37.149705+0000","last_peered":"2024-09-10T18:53:37.149705+0000","last_clean":"2024-09-10T18:53:37.149705+0000","last_became_active":"2024-09-10T18:53:28.298021+0000","last_became_peered":"2024-09-10T18:53:28.298021+0000","last_unstale":"2024-09-10T18:53:37.149705+0000","last_undegraded":"2024-09-10T18:53:37.149705+0000","last_fullsized":"2024-09-10T18:53:37.149705+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:58:04.217092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,7],"acting":[1,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148116+0000","last_change":"2024-09-10T18:53:30.316227+0000","last_active":"2024-09-10T18:53:37.148116+0000","last_peered":"2024-09-10T18:53:37.148116+0000","last_clean":"2024-09-10T18:53:37.148116+0000","last_became_active":"2024-09-10T18:53:30.315935+0000","last_became_peered":"2024-09-10T18:53:30.315935+0000","last_unstale":"2024-09-10T18:53:37.148116+0000","last_undegraded":"2024-09-10T18:53:37.148116+0000","last_fullsized":"2024-09-10T18:53:37.148116+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:50:24.248939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,1],"acting":[4,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435294+0000","last_change":"2024-09-10T18:53:24.323330+0000","last_active":"2024-09-10T18:53:37.435294+0000","last_peered":"2024-09-10T18:53:37.435294+0000","last_clean":"2024-09-10T18:53:37.435294+0000","last_became_active":"2024-09-10T18:53:24.323050+0000","last_became_peered":"2024-09-10T18:53:24.323050+0000","last_unstale":"2024-09-10T18:53:37.435294+0000","last_undegraded":"2024-09-10T18:53:37.435294+0000","last_fullsized":"2024-09-10T18:53:37.435294+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:34:37.847644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,0],"acting":[7,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"59'5","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148222+0000","last_change":"2024-09-10T18:53:26.294016+0000","last_active":"2024-09-10T18:53:37.148222+0000","last_peered":"2024-09-10T18:53:37.148222+0000","last_clean":"2024-09-10T18:53:37.148222+0000","last_became_active":"2024-09-10T18:53:26.293510+0000","last_became_peered":"2024-09-10T18:53:26.293510+0000","last_unstale":"2024-09-10T18:53:37.148222+0000","last_undegraded":"2024-09-10T18:53:37.148222+0000","last_fullsized":"2024-09-10T18:53:37.148222+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:54:54.773423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":7,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,1],"acting":[4,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435048+0000","last_change":"2024-09-10T18:53:28.296981+0000","last_active":"2024-09-10T18:53:37.435048+0000","last_peered":"2024-09-10T18:53:37.435048+0000","last_clean":"2024-09-10T18:53:37.435048+0000","last_became_active":"2024-09-10T18:53:28.296682+0000","last_became_peered":"2024-09-10T18:53:28.296682+0000","last_unstale":"2024-09-10T18:53:37.435048+0000","last_undegraded":"2024-09-10T18:53:37.435048+0000","last_fullsized":"2024-09-10T18:53:37.435048+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:57:57.859622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,1],"acting":[7,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.19","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434320+0000","last_change":"2024-09-10T18:53:30.318079+0000","last_active":"2024-09-10T18:53:37.434320+0000","last_peered":"2024-09-10T18:53:37.434320+0000","last_clean":"2024-09-10T18:53:37.434320+0000","last_became_active":"2024-09-10T18:53:30.317459+0000","last_became_peered":"2024-09-10T18:53:30.317459+0000","last_unstale":"2024-09-10T18:53:37.434320+0000","last_undegraded":"2024-09-10T18:53:37.434320+0000","last_fullsized":"2024-09-10T18:53:37.434320+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:30:21.154537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":13,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,3],"acting":[5,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435704+0000","last_change":"2024-09-10T18:53:24.320047+0000","last_active":"2024-09-10T18:53:37.435704+0000","last_peered":"2024-09-10T18:53:37.435704+0000","last_clean":"2024-09-10T18:53:37.435704+0000","last_became_active":"2024-09-10T18:53:24.319698+0000","last_became_peered":"2024-09-10T18:53:24.319698+0000","last_unstale":"2024-09-10T18:53:37.435704+0000","last_undegraded":"2024-09-10T18:53:37.435704+0000","last_fullsized":"2024-09-10T18:53:37.435704+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:39:06.255321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1a","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148340+0000","last_change":"2024-09-10T18:53:26.298680+0000","last_active":"2024-09-10T18:53:37.148340+0000","last_peered":"2024-09-10T18:53:37.148340+0000","last_clean":"2024-09-10T18:53:37.148340+0000","last_became_active":"2024-09-10T18:53:26.298370+0000","last_became_peered":"2024-09-10T18:53:26.298370+0000","last_unstale":"2024-09-10T18:53:37.148340+0000","last_undegraded":"2024-09-10T18:53:37.148340+0000","last_fullsized":"2024-09-10T18:53:37.148340+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:38:14.386955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,0],"acting":[4,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434677+0000","last_change":"2024-09-10T18:53:28.303526+0000","last_active":"2024-09-10T18:53:37.434677+0000","last_peered":"2024-09-10T18:53:37.434677+0000","last_clean":"2024-09-10T18:53:37.434677+0000","last_became_active":"2024-09-10T18:53:28.303211+0000","last_became_peered":"2024-09-10T18:53:28.303211+0000","last_unstale":"2024-09-10T18:53:37.434677+0000","last_undegraded":"2024-09-10T18:53:37.434677+0000","last_fullsized":"2024-09-10T18:53:37.434677+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:39.302350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,7],"acting":[5,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150968+0000","last_change":"2024-09-10T18:53:30.316863+0000","last_active":"2024-09-10T18:53:37.150968+0000","last_peered":"2024-09-10T18:53:37.150968+0000","last_clean":"2024-09-10T18:53:37.150968+0000","last_became_active":"2024-09-10T18:53:30.316591+0000","last_became_peered":"2024-09-10T18:53:30.316591+0000","last_unstale":"2024-09-10T18:53:37.150968+0000","last_undegraded":"2024-09-10T18:53:37.150968+0000","last_fullsized":"2024-09-10T18:53:37.150968+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:26:18.392905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,7],"acting":[0,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.d","version":"59'17","reported_seq":44,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148811+0000","last_change":"2024-09-10T18:53:26.296532+0000","last_active":"2024-09-10T18:53:37.148811+0000","last_peered":"2024-09-10T18:53:37.148811+0000","last_clean":"2024-09-10T18:53:37.148811+0000","last_became_active":"2024-09-10T18:53:26.296094+0000","last_became_peered":"2024-09-10T18:53:26.296094+0000","last_unstale":"2024-09-10T18:53:37.148811+0000","last_undegraded":"2024-09-10T18:53:37.148811+0000","last_fullsized":"2024-09-10T18:53:37.148811+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:08.669913+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":29,"num_read_kb":19,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,1],"acting":[4,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435099+0000","last_change":"2024-09-10T18:53:24.322438+0000","last_active":"2024-09-10T18:53:37.435099+0000","last_peered":"2024-09-10T18:53:37.435099+0000","last_clean":"2024-09-10T18:53:37.435099+0000","last_became_active":"2024-09-10T18:53:24.322159+0000","last_became_peered":"2024-09-10T18:53:24.322159+0000","last_unstale":"2024-09-10T18:53:37.435099+0000","last_undegraded":"2024-09-10T18:53:37.435099+0000","last_fullsized":"2024-09-10T18:53:37.435099+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:52:14.283289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,5],"acting":[7,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149895+0000","last_change":"2024-09-10T18:53:28.300445+0000","last_active":"2024-09-10T18:53:37.149895+0000","last_peered":"2024-09-10T18:53:37.149895+0000","last_clean":"2024-09-10T18:53:37.149895+0000","last_became_active":"2024-09-10T18:53:28.300177+0000","last_became_peered":"2024-09-10T18:53:28.300177+0000","last_unstale":"2024-09-10T18:53:37.149895+0000","last_undegraded":"2024-09-10T18:53:37.149895+0000","last_fullsized":"2024-09-10T18:53:37.149895+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:01:07.545834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434853+0000","last_change":"2024-09-10T18:53:30.322204+0000","last_active":"2024-09-10T18:53:37.434853+0000","last_peered":"2024-09-10T18:53:37.434853+0000","last_clean":"2024-09-10T18:53:37.434853+0000","last_became_active":"2024-09-10T18:53:30.321833+0000","last_became_peered":"2024-09-10T18:53:30.321833+0000","last_unstale":"2024-09-10T18:53:37.434853+0000","last_undegraded":"2024-09-10T18:53:37.434853+0000","last_fullsized":"2024-09-10T18:53:37.434853+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:54:54.063401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,4],"acting":[2,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148532+0000","last_change":"2024-09-10T18:53:26.297344+0000","last_active":"2024-09-10T18:53:37.148532+0000","last_peered":"2024-09-10T18:53:37.148532+0000","last_clean":"2024-09-10T18:53:37.148532+0000","last_became_active":"2024-09-10T18:53:26.296980+0000","last_became_peered":"2024-09-10T18:53:26.296980+0000","last_unstale":"2024-09-10T18:53:37.148532+0000","last_undegraded":"2024-09-10T18:53:37.148532+0000","last_fullsized":"2024-09-10T18:53:37.148532+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:16:36.750529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,6],"acting":[4,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150580+0000","last_change":"2024-09-10T18:53:24.287714+0000","last_active":"2024-09-10T18:53:37.150580+0000","last_peered":"2024-09-10T18:53:37.150580+0000","last_clean":"2024-09-10T18:53:37.150580+0000","last_became_active":"2024-09-10T18:53:24.287418+0000","last_became_peered":"2024-09-10T18:53:24.287418+0000","last_unstale":"2024-09-10T18:53:37.150580+0000","last_undegraded":"2024-09-10T18:53:37.150580+0000","last_fullsized":"2024-09-10T18:53:37.150580+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:48:01.629263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,7],"acting":[1,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.d","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542657+0000","last_change":"2024-09-10T18:53:28.306463+0000","last_active":"2024-09-10T18:54:37.542657+0000","last_peered":"2024-09-10T18:54:37.542657+0000","last_clean":"2024-09-10T18:54:37.542657+0000","last_became_active":"2024-09-10T18:53:28.305935+0000","last_became_peered":"2024-09-10T18:53:28.305935+0000","last_unstale":"2024-09-10T18:54:37.542657+0000","last_undegraded":"2024-09-10T18:54:37.542657+0000","last_fullsized":"2024-09-10T18:54:37.542657+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:58:46.561887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7,5],"acting":[2,7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148614+0000","last_change":"2024-09-10T18:53:30.318997+0000","last_active":"2024-09-10T18:53:37.148614+0000","last_peered":"2024-09-10T18:53:37.148614+0000","last_clean":"2024-09-10T18:53:37.148614+0000","last_became_active":"2024-09-10T18:53:30.318735+0000","last_became_peered":"2024-09-10T18:53:30.318735+0000","last_unstale":"2024-09-10T18:53:37.148614+0000","last_undegraded":"2024-09-10T18:53:37.148614+0000","last_fullsized":"2024-09-10T18:53:37.148614+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:47:16.108717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2],"acting":[4,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150194+0000","last_change":"2024-09-10T18:53:26.301372+0000","last_active":"2024-09-10T18:53:37.150194+0000","last_peered":"2024-09-10T18:53:37.150194+0000","last_clean":"2024-09-10T18:53:37.150194+0000","last_became_active":"2024-09-10T18:53:26.300708+0000","last_became_peered":"2024-09-10T18:53:26.300708+0000","last_unstale":"2024-09-10T18:53:37.150194+0000","last_undegraded":"2024-09-10T18:53:37.150194+0000","last_fullsized":"2024-09-10T18:53:37.150194+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:03:00.455320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3,4],"acting":[1,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.9","version":"52'1","reported_seq":26,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149798+0000","last_change":"2024-09-10T18:53:24.287264+0000","last_active":"2024-09-10T18:53:37.149798+0000","last_peered":"2024-09-10T18:53:37.149798+0000","last_clean":"2024-09-10T18:53:37.149798+0000","last_became_active":"2024-09-10T18:53:24.286916+0000","last_became_peered":"2024-09-10T18:53:24.286916+0000","last_unstale":"2024-09-10T18:53:37.149798+0000","last_undegraded":"2024-09-10T18:53:37.149798+0000","last_fullsized":"2024-09-10T18:53:37.149798+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:21.363101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":874,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.e","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543082+0000","last_change":"2024-09-10T18:53:28.307959+0000","last_active":"2024-09-10T18:54:37.543082+0000","last_peered":"2024-09-10T18:54:37.543082+0000","last_clean":"2024-09-10T18:54:37.543082+0000","last_became_active":"2024-09-10T18:53:28.307578+0000","last_became_peered":"2024-09-10T18:53:28.307578+0000","last_unstale":"2024-09-10T18:54:37.543082+0000","last_undegraded":"2024-09-10T18:54:37.543082+0000","last_fullsized":"2024-09-10T18:54:37.543082+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:53:11.692163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,0],"acting":[4,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434467+0000","last_change":"2024-09-10T18:53:30.322430+0000","last_active":"2024-09-10T18:53:37.434467+0000","last_peered":"2024-09-10T18:53:37.434467+0000","last_clean":"2024-09-10T18:53:37.434467+0000","last_became_active":"2024-09-10T18:53:30.322136+0000","last_became_peered":"2024-09-10T18:53:30.322136+0000","last_unstale":"2024-09-10T18:53:37.434467+0000","last_undegraded":"2024-09-10T18:53:37.434467+0000","last_fullsized":"2024-09-10T18:53:37.434467+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:31:11.215386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148726+0000","last_change":"2024-09-10T18:53:26.296824+0000","last_active":"2024-09-10T18:53:37.148726+0000","last_peered":"2024-09-10T18:53:37.148726+0000","last_clean":"2024-09-10T18:53:37.148726+0000","last_became_active":"2024-09-10T18:53:26.296378+0000","last_became_peered":"2024-09-10T18:53:26.296378+0000","last_unstale":"2024-09-10T18:53:37.148726+0000","last_undegraded":"2024-09-10T18:53:37.148726+0000","last_fullsized":"2024-09-10T18:53:37.148726+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:00:53.035182+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434853+0000","last_change":"2024-09-10T18:53:24.323809+0000","last_active":"2024-09-10T18:53:37.434853+0000","last_peered":"2024-09-10T18:53:37.434853+0000","last_clean":"2024-09-10T18:53:37.434853+0000","last_became_active":"2024-09-10T18:53:24.323101+0000","last_became_peered":"2024-09-10T18:53:24.323101+0000","last_unstale":"2024-09-10T18:53:37.434853+0000","last_undegraded":"2024-09-10T18:53:37.434853+0000","last_fullsized":"2024-09-10T18:53:37.434853+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:08:44.726626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,1],"acting":[7,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435720+0000","last_change":"2024-09-10T18:53:28.302549+0000","last_active":"2024-09-10T18:53:37.435720+0000","last_peered":"2024-09-10T18:53:37.435720+0000","last_clean":"2024-09-10T18:53:37.435720+0000","last_became_active":"2024-09-10T18:53:28.301984+0000","last_became_peered":"2024-09-10T18:53:28.301984+0000","last_unstale":"2024-09-10T18:53:37.435720+0000","last_undegraded":"2024-09-10T18:53:37.435720+0000","last_fullsized":"2024-09-10T18:53:37.435720+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:51:15.220360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,6],"acting":[5,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434751+0000","last_change":"2024-09-10T18:53:30.493864+0000","last_active":"2024-09-10T18:53:37.434751+0000","last_peered":"2024-09-10T18:53:37.434751+0000","last_clean":"2024-09-10T18:53:37.434751+0000","last_became_active":"2024-09-10T18:53:30.493550+0000","last_became_peered":"2024-09-10T18:53:30.493550+0000","last_unstale":"2024-09-10T18:53:37.434751+0000","last_undegraded":"2024-09-10T18:53:37.434751+0000","last_fullsized":"2024-09-10T18:53:37.434751+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:38.229848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"59'14","reported_seq":37,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148178+0000","last_change":"2024-09-10T18:53:26.298164+0000","last_active":"2024-09-10T18:53:37.148178+0000","last_peered":"2024-09-10T18:53:37.148178+0000","last_clean":"2024-09-10T18:53:37.148178+0000","last_became_active":"2024-09-10T18:53:26.297438+0000","last_became_peered":"2024-09-10T18:53:26.297438+0000","last_unstale":"2024-09-10T18:53:37.148178+0000","last_undegraded":"2024-09-10T18:53:37.148178+0000","last_fullsized":"2024-09-10T18:53:37.148178+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:56:33.416772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":14,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,6],"acting":[4,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363534+0000","last_change":"2024-09-10T18:53:24.318909+0000","last_active":"2024-09-10T18:53:37.363534+0000","last_peered":"2024-09-10T18:53:37.363534+0000","last_clean":"2024-09-10T18:53:37.363534+0000","last_became_active":"2024-09-10T18:53:24.318518+0000","last_became_peered":"2024-09-10T18:53:24.318518+0000","last_unstale":"2024-09-10T18:53:37.363534+0000","last_undegraded":"2024-09-10T18:53:37.363534+0000","last_fullsized":"2024-09-10T18:53:37.363534+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:13:53.887357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,2],"acting":[6,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.0","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542695+0000","last_change":"2024-09-10T18:53:28.311491+0000","last_active":"2024-09-10T18:54:37.542695+0000","last_peered":"2024-09-10T18:54:37.542695+0000","last_clean":"2024-09-10T18:54:37.542695+0000","last_became_active":"2024-09-10T18:53:28.310609+0000","last_became_peered":"2024-09-10T18:53:28.310609+0000","last_unstale":"2024-09-10T18:54:37.542695+0000","last_undegraded":"2024-09-10T18:54:37.542695+0000","last_fullsized":"2024-09-10T18:54:37.542695+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:14:46.705934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434927+0000","last_change":"2024-09-10T18:53:30.494894+0000","last_active":"2024-09-10T18:53:37.434927+0000","last_peered":"2024-09-10T18:53:37.434927+0000","last_clean":"2024-09-10T18:53:37.434927+0000","last_became_active":"2024-09-10T18:53:30.494124+0000","last_became_peered":"2024-09-10T18:53:30.494124+0000","last_unstale":"2024-09-10T18:53:37.434927+0000","last_undegraded":"2024-09-10T18:53:37.434927+0000","last_fullsized":"2024-09-10T18:53:37.434927+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:37:27.871833+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,2],"acting":[7,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.0","version":"59'18","reported_seq":48,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434270+0000","last_change":"2024-09-10T18:53:26.436695+0000","last_active":"2024-09-10T18:53:37.434270+0000","last_peered":"2024-09-10T18:53:37.434270+0000","last_clean":"2024-09-10T18:53:37.434270+0000","last_became_active":"2024-09-10T18:53:26.436312+0000","last_became_peered":"2024-09-10T18:53:26.436312+0000","last_unstale":"2024-09-10T18:53:37.434270+0000","last_undegraded":"2024-09-10T18:53:37.434270+0000","last_fullsized":"2024-09-10T18:53:37.434270+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:15:05.878924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":34,"num_read_kb":22,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,0],"acting":[3,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150028+0000","last_change":"2024-09-10T18:53:24.285419+0000","last_active":"2024-09-10T18:53:37.150028+0000","last_peered":"2024-09-10T18:53:37.150028+0000","last_clean":"2024-09-10T18:53:37.150028+0000","last_became_active":"2024-09-10T18:53:24.285142+0000","last_became_peered":"2024-09-10T18:53:24.285142+0000","last_unstale":"2024-09-10T18:53:37.150028+0000","last_undegraded":"2024-09-10T18:53:37.150028+0000","last_fullsized":"2024-09-10T18:53:37.150028+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:46.080848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,4],"acting":[1,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148054+0000","last_change":"2024-09-10T18:53:28.306671+0000","last_active":"2024-09-10T18:53:37.148054+0000","last_peered":"2024-09-10T18:53:37.148054+0000","last_clean":"2024-09-10T18:53:37.148054+0000","last_became_active":"2024-09-10T18:53:28.306356+0000","last_became_peered":"2024-09-10T18:53:28.306356+0000","last_unstale":"2024-09-10T18:53:37.148054+0000","last_undegraded":"2024-09-10T18:53:37.148054+0000","last_fullsized":"2024-09-10T18:53:37.148054+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:25:19.368023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,7],"acting":[4,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148275+0000","last_change":"2024-09-10T18:53:30.318027+0000","last_active":"2024-09-10T18:53:37.148275+0000","last_peered":"2024-09-10T18:53:37.148275+0000","last_clean":"2024-09-10T18:53:37.148275+0000","last_became_active":"2024-09-10T18:53:30.317760+0000","last_became_peered":"2024-09-10T18:53:30.317760+0000","last_unstale":"2024-09-10T18:53:37.148275+0000","last_undegraded":"2024-09-10T18:53:37.148275+0000","last_fullsized":"2024-09-10T18:53:37.148275+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:08:52.895640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3","version":"59'19","reported_seq":52,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151530+0000","last_change":"2024-09-10T18:53:26.436852+0000","last_active":"2024-09-10T18:53:37.151530+0000","last_peered":"2024-09-10T18:53:37.151530+0000","last_clean":"2024-09-10T18:53:37.151530+0000","last_became_active":"2024-09-10T18:53:26.436347+0000","last_became_peered":"2024-09-10T18:53:26.436347+0000","last_unstale":"2024-09-10T18:53:37.151530+0000","last_undegraded":"2024-09-10T18:53:37.151530+0000","last_fullsized":"2024-09-10T18:53:37.151530+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:20:47.996764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":39,"num_read_kb":25,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,7],"acting":[0,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435323+0000","last_change":"2024-09-10T18:53:24.323366+0000","last_active":"2024-09-10T18:53:37.435323+0000","last_peered":"2024-09-10T18:53:37.435323+0000","last_clean":"2024-09-10T18:53:37.435323+0000","last_became_active":"2024-09-10T18:53:24.323123+0000","last_became_peered":"2024-09-10T18:53:24.323123+0000","last_unstale":"2024-09-10T18:53:37.435323+0000","last_undegraded":"2024-09-10T18:53:37.435323+0000","last_fullsized":"2024-09-10T18:53:37.435323+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:09:54.514253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,4],"acting":[7,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.362951+0000","last_change":"2024-09-10T18:53:28.303903+0000","last_active":"2024-09-10T18:53:37.362951+0000","last_peered":"2024-09-10T18:53:37.362951+0000","last_clean":"2024-09-10T18:53:37.362951+0000","last_became_active":"2024-09-10T18:53:28.303638+0000","last_became_peered":"2024-09-10T18:53:28.303638+0000","last_unstale":"2024-09-10T18:53:37.362951+0000","last_undegraded":"2024-09-10T18:53:37.362951+0000","last_fullsized":"2024-09-10T18:53:37.362951+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:24:32.062951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,5],"acting":[6,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149663+0000","last_change":"2024-09-10T18:53:30.493954+0000","last_active":"2024-09-10T18:53:37.149663+0000","last_peered":"2024-09-10T18:53:37.149663+0000","last_clean":"2024-09-10T18:53:37.149663+0000","last_became_active":"2024-09-10T18:53:30.493580+0000","last_became_peered":"2024-09-10T18:53:30.493580+0000","last_unstale":"2024-09-10T18:53:37.149663+0000","last_undegraded":"2024-09-10T18:53:37.149663+0000","last_fullsized":"2024-09-10T18:53:37.149663+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:35:36.616675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150503+0000","last_change":"2024-09-10T18:53:26.301783+0000","last_active":"2024-09-10T18:53:37.150503+0000","last_peered":"2024-09-10T18:53:37.150503+0000","last_clean":"2024-09-10T18:53:37.150503+0000","last_became_active":"2024-09-10T18:53:26.301155+0000","last_became_peered":"2024-09-10T18:53:26.301155+0000","last_unstale":"2024-09-10T18:53:37.150503+0000","last_undegraded":"2024-09-10T18:53:37.150503+0000","last_fullsized":"2024-09-10T18:53:37.150503+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:38.192190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150234+0000","last_change":"2024-09-10T18:53:24.288264+0000","last_active":"2024-09-10T18:53:37.150234+0000","last_peered":"2024-09-10T18:53:37.150234+0000","last_clean":"2024-09-10T18:53:37.150234+0000","last_became_active":"2024-09-10T18:53:24.287980+0000","last_became_peered":"2024-09-10T18:53:24.287980+0000","last_unstale":"2024-09-10T18:53:37.150234+0000","last_undegraded":"2024-09-10T18:53:37.150234+0000","last_fullsized":"2024-09-10T18:53:37.150234+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:23:50.999623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0,7],"acting":[1,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542661+0000","last_change":"2024-09-10T18:53:28.303638+0000","last_active":"2024-09-10T18:54:37.542661+0000","last_peered":"2024-09-10T18:54:37.542661+0000","last_clean":"2024-09-10T18:54:37.542661+0000","last_became_active":"2024-09-10T18:53:28.303317+0000","last_became_peered":"2024-09-10T18:53:28.303317+0000","last_unstale":"2024-09-10T18:54:37.542661+0000","last_undegraded":"2024-09-10T18:54:37.542661+0000","last_fullsized":"2024-09-10T18:54:37.542661+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:47:59.432446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,5],"acting":[0,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150765+0000","last_change":"2024-09-10T18:53:30.317428+0000","last_active":"2024-09-10T18:53:37.150765+0000","last_peered":"2024-09-10T18:53:37.150765+0000","last_clean":"2024-09-10T18:53:37.150765+0000","last_became_active":"2024-09-10T18:53:30.317136+0000","last_became_peered":"2024-09-10T18:53:30.317136+0000","last_unstale":"2024-09-10T18:53:37.150765+0000","last_undegraded":"2024-09-10T18:53:37.150765+0000","last_fullsized":"2024-09-10T18:53:37.150765+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:36:09.859951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,2],"acting":[0,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"59'28","reported_seq":80,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542656+0000","last_change":"2024-09-10T18:53:26.302071+0000","last_active":"2024-09-10T18:54:37.542656+0000","last_peered":"2024-09-10T18:54:37.542656+0000","last_clean":"2024-09-10T18:54:37.542656+0000","last_became_active":"2024-09-10T18:53:26.301787+0000","last_became_peered":"2024-09-10T18:53:26.301787+0000","last_unstale":"2024-09-10T18:54:37.542656+0000","last_undegraded":"2024-09-10T18:54:37.542656+0000","last_fullsized":"2024-09-10T18:54:37.542656+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:17:52.773006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":10,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":48,"num_read_kb":33,"num_write":26,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,3],"acting":[1,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434865+0000","last_change":"2024-09-10T18:53:24.315960+0000","last_active":"2024-09-10T18:53:37.434865+0000","last_peered":"2024-09-10T18:53:37.434865+0000","last_clean":"2024-09-10T18:53:37.434865+0000","last_became_active":"2024-09-10T18:53:24.315702+0000","last_became_peered":"2024-09-10T18:53:24.315702+0000","last_unstale":"2024-09-10T18:53:37.434865+0000","last_undegraded":"2024-09-10T18:53:37.434865+0000","last_fullsized":"2024-09-10T18:53:37.434865+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:35:41.956660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,6],"acting":[5,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151414+0000","last_change":"2024-09-10T18:53:28.304974+0000","last_active":"2024-09-10T18:53:37.151414+0000","last_peered":"2024-09-10T18:53:37.151414+0000","last_clean":"2024-09-10T18:53:37.151414+0000","last_became_active":"2024-09-10T18:53:28.304601+0000","last_became_peered":"2024-09-10T18:53:28.304601+0000","last_unstale":"2024-09-10T18:53:37.151414+0000","last_undegraded":"2024-09-10T18:53:37.151414+0000","last_fullsized":"2024-09-10T18:53:37.151414+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:51.997529+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435038+0000","last_change":"2024-09-10T18:53:30.319419+0000","last_active":"2024-09-10T18:53:37.435038+0000","last_peered":"2024-09-10T18:53:37.435038+0000","last_clean":"2024-09-10T18:53:37.435038+0000","last_became_active":"2024-09-10T18:53:30.319147+0000","last_became_peered":"2024-09-10T18:53:30.319147+0000","last_unstale":"2024-09-10T18:53:37.435038+0000","last_undegraded":"2024-09-10T18:53:37.435038+0000","last_fullsized":"2024-09-10T18:53:37.435038+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:14:34.252357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,7],"acting":[3,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.7","version":"59'13","reported_seq":43,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150212+0000","last_change":"2024-09-10T18:53:26.303246+0000","last_active":"2024-09-10T18:53:37.150212+0000","last_peered":"2024-09-10T18:53:37.150212+0000","last_clean":"2024-09-10T18:53:37.150212+0000","last_became_active":"2024-09-10T18:53:26.302458+0000","last_became_peered":"2024-09-10T18:53:26.302458+0000","last_unstale":"2024-09-10T18:53:37.150212+0000","last_undegraded":"2024-09-10T18:53:37.150212+0000","last_fullsized":"2024-09-10T18:53:37.150212+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:59:55.887266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":30,"num_read_kb":19,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.0","version":"59'5","reported_seq":102,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:45.727202+0000","last_change":"2024-09-10T18:53:28.290511+0000","last_active":"2024-09-10T18:54:45.727202+0000","last_peered":"2024-09-10T18:54:45.727202+0000","last_clean":"2024-09-10T18:54:45.727202+0000","last_became_active":"2024-09-10T18:53:24.284144+0000","last_became_peered":"2024-09-10T18:53:24.284144+0000","last_unstale":"2024-09-10T18:54:45.727202+0000","last_undegraded":"2024-09-10T18:54:45.727202+0000","last_fullsized":"2024-09-10T18:54:45.727202+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:04:45.531721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.00051018899999999998,"stat_sum":{"num_bytes":389,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,6],"acting":[1,2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434613+0000","last_change":"2024-09-10T18:53:24.289353+0000","last_active":"2024-09-10T18:53:37.434613+0000","last_peered":"2024-09-10T18:53:37.434613+0000","last_clean":"2024-09-10T18:53:37.434613+0000","last_became_active":"2024-09-10T18:53:24.289052+0000","last_became_peered":"2024-09-10T18:53:24.289052+0000","last_unstale":"2024-09-10T18:53:37.434613+0000","last_undegraded":"2024-09-10T18:53:37.434613+0000","last_fullsized":"2024-09-10T18:53:37.434613+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T18:55:25.959436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435053+0000","last_change":"2024-09-10T18:53:28.299521+0000","last_active":"2024-09-10T18:53:37.435053+0000","last_peered":"2024-09-10T18:53:37.435053+0000","last_clean":"2024-09-10T18:53:37.435053+0000","last_became_active":"2024-09-10T18:53:28.299223+0000","last_became_peered":"2024-09-10T18:53:28.299223+0000","last_unstale":"2024-09-10T18:53:37.435053+0000","last_undegraded":"2024-09-10T18:53:37.435053+0000","last_fullsized":"2024-09-10T18:53:37.435053+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:30:38.149327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7],"acting":[2,5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435061+0000","last_change":"2024-09-10T18:53:30.493741+0000","last_active":"2024-09-10T18:53:37.435061+0000","last_peered":"2024-09-10T18:53:37.435061+0000","last_clean":"2024-09-10T18:53:37.435061+0000","last_became_active":"2024-09-10T18:53:30.493434+0000","last_became_peered":"2024-09-10T18:53:30.493434+0000","last_unstale":"2024-09-10T18:53:37.435061+0000","last_undegraded":"2024-09-10T18:53:37.435061+0000","last_fullsized":"2024-09-10T18:53:37.435061+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:23:04.260417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,3],"acting":[7,6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"59'12","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151607+0000","last_change":"2024-09-10T18:53:26.301162+0000","last_active":"2024-09-10T18:53:37.151607+0000","last_peered":"2024-09-10T18:53:37.151607+0000","last_clean":"2024-09-10T18:53:37.151607+0000","last_became_active":"2024-09-10T18:53:26.300871+0000","last_became_peered":"2024-09-10T18:53:26.300871+0000","last_unstale":"2024-09-10T18:53:37.151607+0000","last_undegraded":"2024-09-10T18:53:37.151607+0000","last_fullsized":"2024-09-10T18:53:37.151607+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:57:51.606275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":18,"num_read_kb":12,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,2],"acting":[0,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1","version":"54'1","reported_seq":24,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151131+0000","last_change":"2024-09-10T18:53:28.299930+0000","last_active":"2024-09-10T18:53:37.151131+0000","last_peered":"2024-09-10T18:53:37.151131+0000","last_clean":"2024-09-10T18:53:37.151131+0000","last_became_active":"2024-09-10T18:53:24.289641+0000","last_became_peered":"2024-09-10T18:53:24.289641+0000","last_unstale":"2024-09-10T18:53:37.151131+0000","last_undegraded":"2024-09-10T18:53:37.151131+0000","last_fullsized":"2024-09-10T18:53:37.151131+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:04:16.222970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.002086638,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434722+0000","last_change":"2024-09-10T18:53:24.323365+0000","last_active":"2024-09-10T18:53:37.434722+0000","last_peered":"2024-09-10T18:53:37.434722+0000","last_clean":"2024-09-10T18:53:37.434722+0000","last_became_active":"2024-09-10T18:53:24.323022+0000","last_became_peered":"2024-09-10T18:53:24.323022+0000","last_unstale":"2024-09-10T18:53:37.434722+0000","last_undegraded":"2024-09-10T18:53:37.434722+0000","last_fullsized":"2024-09-10T18:53:37.434722+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:50:09.795084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,0],"acting":[7,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435804+0000","last_change":"2024-09-10T18:53:28.301432+0000","last_active":"2024-09-10T18:53:37.435804+0000","last_peered":"2024-09-10T18:53:37.435804+0000","last_clean":"2024-09-10T18:53:37.435804+0000","last_became_active":"2024-09-10T18:53:28.301153+0000","last_became_peered":"2024-09-10T18:53:28.301153+0000","last_unstale":"2024-09-10T18:53:37.435804+0000","last_undegraded":"2024-09-10T18:53:37.435804+0000","last_fullsized":"2024-09-10T18:53:37.435804+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:16:06.052714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.149816+0000","last_change":"2024-09-10T18:53:30.316591+0000","last_active":"2024-09-10T18:53:37.149816+0000","last_peered":"2024-09-10T18:53:37.149816+0000","last_clean":"2024-09-10T18:53:37.149816+0000","last_became_active":"2024-09-10T18:53:30.316280+0000","last_became_peered":"2024-09-10T18:53:30.316280+0000","last_unstale":"2024-09-10T18:53:37.149816+0000","last_undegraded":"2024-09-10T18:53:37.149816+0000","last_fullsized":"2024-09-10T18:53:37.149816+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:17:02.570839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"59'14","reported_seq":38,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363345+0000","last_change":"2024-09-10T18:53:26.435625+0000","last_active":"2024-09-10T18:53:37.363345+0000","last_peered":"2024-09-10T18:53:37.363345+0000","last_clean":"2024-09-10T18:53:37.363345+0000","last_became_active":"2024-09-10T18:53:26.435316+0000","last_became_peered":"2024-09-10T18:53:26.435316+0000","last_unstale":"2024-09-10T18:53:37.363345+0000","last_undegraded":"2024-09-10T18:53:37.363345+0000","last_fullsized":"2024-09-10T18:53:37.363345+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:31:54.666725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":25,"num_read_kb":15,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2","version":"56'2","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434743+0000","last_change":"2024-09-10T18:53:28.302455+0000","last_active":"2024-09-10T18:53:37.434743+0000","last_peered":"2024-09-10T18:53:37.434743+0000","last_clean":"2024-09-10T18:53:37.434743+0000","last_became_active":"2024-09-10T18:53:24.318491+0000","last_became_peered":"2024-09-10T18:53:24.318491+0000","last_unstale":"2024-09-10T18:53:37.434743+0000","last_undegraded":"2024-09-10T18:53:37.434743+0000","last_fullsized":"2024-09-10T18:53:37.434743+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:49:49.320134+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0.0012118369999999999,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,6],"acting":[3,5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434877+0000","last_change":"2024-09-10T18:53:24.317399+0000","last_active":"2024-09-10T18:53:37.434877+0000","last_peered":"2024-09-10T18:53:37.434877+0000","last_clean":"2024-09-10T18:53:37.434877+0000","last_became_active":"2024-09-10T18:53:24.316221+0000","last_became_peered":"2024-09-10T18:53:24.316221+0000","last_unstale":"2024-09-10T18:53:37.434877+0000","last_undegraded":"2024-09-10T18:53:37.434877+0000","last_fullsized":"2024-09-10T18:53:37.434877+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:09:15.626834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"61'79","reported_seq":52,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:03.080376+0000","last_change":"2024-09-10T18:53:12.962331+0000","last_active":"2024-09-10T18:54:03.080376+0000","last_peered":"2024-09-10T18:54:03.080376+0000","last_clean":"2024-09-10T18:54:03.080376+0000","last_became_active":"2024-09-10T18:53:12.956704+0000","last_became_peered":"2024-09-10T18:53:12.956704+0000","last_unstale":"2024-09-10T18:54:03.080376+0000","last_undegraded":"2024-09-10T18:54:03.080376+0000","last_fullsized":"2024-09-10T18:54:03.080376+0000","mapping_epoch":49,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":50,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:51:13.632166+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:51:13.632166+0000","last_clean_scrub_stamp":"2024-09-10T18:51:13.632166+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:35:41.684726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0,6],"acting":[7,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435522+0000","last_change":"2024-09-10T18:53:28.295888+0000","last_active":"2024-09-10T18:53:37.435522+0000","last_peered":"2024-09-10T18:53:37.435522+0000","last_clean":"2024-09-10T18:53:37.435522+0000","last_became_active":"2024-09-10T18:53:28.295573+0000","last_became_peered":"2024-09-10T18:53:28.295573+0000","last_unstale":"2024-09-10T18:53:37.435522+0000","last_undegraded":"2024-09-10T18:53:37.435522+0000","last_fullsized":"2024-09-10T18:53:37.435522+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:15:37.312150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,5],"acting":[7,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435821+0000","last_change":"2024-09-10T18:53:30.320283+0000","last_active":"2024-09-10T18:53:37.435821+0000","last_peered":"2024-09-10T18:53:37.435821+0000","last_clean":"2024-09-10T18:53:37.435821+0000","last_became_active":"2024-09-10T18:53:30.319653+0000","last_became_peered":"2024-09-10T18:53:30.319653+0000","last_unstale":"2024-09-10T18:53:37.435821+0000","last_undegraded":"2024-09-10T18:53:37.435821+0000","last_fullsized":"2024-09-10T18:53:37.435821+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:47:53.659336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,4],"acting":[5,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.a","version":"59'19","reported_seq":47,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363197+0000","last_change":"2024-09-10T18:53:26.435558+0000","last_active":"2024-09-10T18:53:37.363197+0000","last_peered":"2024-09-10T18:53:37.363197+0000","last_clean":"2024-09-10T18:53:37.363197+0000","last_became_active":"2024-09-10T18:53:26.434916+0000","last_became_peered":"2024-09-10T18:53:26.434916+0000","last_unstale":"2024-09-10T18:53:37.363197+0000","last_undegraded":"2024-09-10T18:53:37.363197+0000","last_fullsized":"2024-09-10T18:53:37.363197+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":19,"log_dups_size":0,"ondisk_log_size":19,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:35:08.784877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":27,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":32,"num_read_kb":21,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1,7],"acting":[6,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435453+0000","last_change":"2024-09-10T18:53:24.286621+0000","last_active":"2024-09-10T18:53:37.435453+0000","last_peered":"2024-09-10T18:53:37.435453+0000","last_clean":"2024-09-10T18:53:37.435453+0000","last_became_active":"2024-09-10T18:53:24.286301+0000","last_became_peered":"2024-09-10T18:53:24.286301+0000","last_unstale":"2024-09-10T18:53:37.435453+0000","last_undegraded":"2024-09-10T18:53:37.435453+0000","last_fullsized":"2024-09-10T18:53:37.435453+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:53.791835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,0],"acting":[2,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434854+0000","last_change":"2024-09-10T18:53:28.306824+0000","last_active":"2024-09-10T18:53:37.434854+0000","last_peered":"2024-09-10T18:53:37.434854+0000","last_clean":"2024-09-10T18:53:37.434854+0000","last_became_active":"2024-09-10T18:53:28.306491+0000","last_became_peered":"2024-09-10T18:53:28.306491+0000","last_unstale":"2024-09-10T18:53:37.434854+0000","last_undegraded":"2024-09-10T18:53:37.434854+0000","last_fullsized":"2024-09-10T18:53:37.434854+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:51.444075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,5],"acting":[2,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434799+0000","last_change":"2024-09-10T18:53:30.324145+0000","last_active":"2024-09-10T18:53:37.434799+0000","last_peered":"2024-09-10T18:53:37.434799+0000","last_clean":"2024-09-10T18:53:37.434799+0000","last_became_active":"2024-09-10T18:53:30.323941+0000","last_became_peered":"2024-09-10T18:53:30.323941+0000","last_unstale":"2024-09-10T18:53:37.434799+0000","last_undegraded":"2024-09-10T18:53:37.434799+0000","last_fullsized":"2024-09-10T18:53:37.434799+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:51:22.303569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,3],"acting":[7,2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151367+0000","last_change":"2024-09-10T18:53:26.303781+0000","last_active":"2024-09-10T18:53:37.151367+0000","last_peered":"2024-09-10T18:53:37.151367+0000","last_clean":"2024-09-10T18:53:37.151367+0000","last_became_active":"2024-09-10T18:53:26.303330+0000","last_became_peered":"2024-09-10T18:53:26.303330+0000","last_unstale":"2024-09-10T18:53:37.151367+0000","last_undegraded":"2024-09-10T18:53:37.151367+0000","last_fullsized":"2024-09-10T18:53:37.151367+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:15:26.699302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,4],"acting":[0,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150382+0000","last_change":"2024-09-10T18:53:24.286522+0000","last_active":"2024-09-10T18:53:37.150382+0000","last_peered":"2024-09-10T18:53:37.150382+0000","last_clean":"2024-09-10T18:53:37.150382+0000","last_became_active":"2024-09-10T18:53:24.286255+0000","last_became_peered":"2024-09-10T18:53:24.286255+0000","last_unstale":"2024-09-10T18:53:37.150382+0000","last_undegraded":"2024-09-10T18:53:37.150382+0000","last_fullsized":"2024-09-10T18:53:37.150382+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:31.711842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,3],"acting":[1,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434726+0000","last_change":"2024-09-10T18:53:28.306325+0000","last_active":"2024-09-10T18:53:37.434726+0000","last_peered":"2024-09-10T18:53:37.434726+0000","last_clean":"2024-09-10T18:53:37.434726+0000","last_became_active":"2024-09-10T18:53:28.305778+0000","last_became_peered":"2024-09-10T18:53:28.305778+0000","last_unstale":"2024-09-10T18:53:37.434726+0000","last_undegraded":"2024-09-10T18:53:37.434726+0000","last_fullsized":"2024-09-10T18:53:37.434726+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:44:17.756972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,3],"acting":[2,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150970+0000","last_change":"2024-09-10T18:53:30.317479+0000","last_active":"2024-09-10T18:53:37.150970+0000","last_peered":"2024-09-10T18:53:37.150970+0000","last_clean":"2024-09-10T18:53:37.150970+0000","last_became_active":"2024-09-10T18:53:30.317187+0000","last_became_peered":"2024-09-10T18:53:30.317187+0000","last_unstale":"2024-09-10T18:53:37.150970+0000","last_undegraded":"2024-09-10T18:53:37.150970+0000","last_fullsized":"2024-09-10T18:53:37.150970+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:14:20.902323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434316+0000","last_change":"2024-09-10T18:53:26.435709+0000","last_active":"2024-09-10T18:53:37.434316+0000","last_peered":"2024-09-10T18:53:37.434316+0000","last_clean":"2024-09-10T18:53:37.434316+0000","last_became_active":"2024-09-10T18:53:26.435438+0000","last_became_peered":"2024-09-10T18:53:26.435438+0000","last_unstale":"2024-09-10T18:53:37.434316+0000","last_undegraded":"2024-09-10T18:53:37.434316+0000","last_fullsized":"2024-09-10T18:53:37.434316+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:42:40.031213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,6],"acting":[5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435224+0000","last_change":"2024-09-10T18:53:24.287837+0000","last_active":"2024-09-10T18:53:37.435224+0000","last_peered":"2024-09-10T18:53:37.435224+0000","last_clean":"2024-09-10T18:53:37.435224+0000","last_became_active":"2024-09-10T18:53:24.287579+0000","last_became_peered":"2024-09-10T18:53:24.287579+0000","last_unstale":"2024-09-10T18:53:37.435224+0000","last_undegraded":"2024-09-10T18:53:37.435224+0000","last_fullsized":"2024-09-10T18:53:37.435224+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:32:20.965123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,7],"acting":[2,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543090+0000","last_change":"2024-09-10T18:53:28.298827+0000","last_active":"2024-09-10T18:54:37.543090+0000","last_peered":"2024-09-10T18:54:37.543090+0000","last_clean":"2024-09-10T18:54:37.543090+0000","last_became_active":"2024-09-10T18:53:28.298559+0000","last_became_peered":"2024-09-10T18:53:28.298559+0000","last_unstale":"2024-09-10T18:54:37.543090+0000","last_undegraded":"2024-09-10T18:54:37.543090+0000","last_fullsized":"2024-09-10T18:54:37.543090+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:06:13.384749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434513+0000","last_change":"2024-09-10T18:53:30.494732+0000","last_active":"2024-09-10T18:53:37.434513+0000","last_peered":"2024-09-10T18:53:37.434513+0000","last_clean":"2024-09-10T18:53:37.434513+0000","last_became_active":"2024-09-10T18:53:30.494397+0000","last_became_peered":"2024-09-10T18:53:30.494397+0000","last_unstale":"2024-09-10T18:53:37.434513+0000","last_undegraded":"2024-09-10T18:53:37.434513+0000","last_fullsized":"2024-09-10T18:53:37.434513+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:20:27.019655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,0],"acting":[5,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148060+0000","last_change":"2024-09-10T18:53:26.291082+0000","last_active":"2024-09-10T18:53:37.148060+0000","last_peered":"2024-09-10T18:53:37.148060+0000","last_clean":"2024-09-10T18:53:37.148060+0000","last_became_active":"2024-09-10T18:53:26.290878+0000","last_became_peered":"2024-09-10T18:53:26.290878+0000","last_unstale":"2024-09-10T18:53:37.148060+0000","last_undegraded":"2024-09-10T18:53:37.148060+0000","last_fullsized":"2024-09-10T18:53:37.148060+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:56:43.920371+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,3],"acting":[4,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.f","version":"52'2","reported_seq":34,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148921+0000","last_change":"2024-09-10T18:53:24.286439+0000","last_active":"2024-09-10T18:53:37.148921+0000","last_peered":"2024-09-10T18:53:37.148921+0000","last_clean":"2024-09-10T18:53:37.148921+0000","last_became_active":"2024-09-10T18:53:24.286198+0000","last_became_peered":"2024-09-10T18:53:24.286198+0000","last_unstale":"2024-09-10T18:53:37.148921+0000","last_undegraded":"2024-09-10T18:53:37.148921+0000","last_fullsized":"2024-09-10T18:53:37.148921+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:55:27.304101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0,7],"acting":[4,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434289+0000","last_change":"2024-09-10T18:53:28.302633+0000","last_active":"2024-09-10T18:53:37.434289+0000","last_peered":"2024-09-10T18:53:37.434289+0000","last_clean":"2024-09-10T18:53:37.434289+0000","last_became_active":"2024-09-10T18:53:28.299141+0000","last_became_peered":"2024-09-10T18:53:28.299141+0000","last_unstale":"2024-09-10T18:53:37.434289+0000","last_undegraded":"2024-09-10T18:53:37.434289+0000","last_fullsized":"2024-09-10T18:53:37.434289+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:50:45.400311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0,1],"acting":[2,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434336+0000","last_change":"2024-09-10T18:53:30.319194+0000","last_active":"2024-09-10T18:53:37.434336+0000","last_peered":"2024-09-10T18:53:37.434336+0000","last_clean":"2024-09-10T18:53:37.434336+0000","last_became_active":"2024-09-10T18:53:30.318898+0000","last_became_peered":"2024-09-10T18:53:30.318898+0000","last_unstale":"2024-09-10T18:53:37.434336+0000","last_undegraded":"2024-09-10T18:53:37.434336+0000","last_fullsized":"2024-09-10T18:53:37.434336+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:29:06.510757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151462+0000","last_change":"2024-09-10T18:53:26.437618+0000","last_active":"2024-09-10T18:53:37.151462+0000","last_peered":"2024-09-10T18:53:37.151462+0000","last_clean":"2024-09-10T18:53:37.151462+0000","last_became_active":"2024-09-10T18:53:26.437316+0000","last_became_peered":"2024-09-10T18:53:26.437316+0000","last_unstale":"2024-09-10T18:53:37.151462+0000","last_undegraded":"2024-09-10T18:53:37.151462+0000","last_fullsized":"2024-09-10T18:53:37.151462+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:27:17.346745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,7],"acting":[0,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.10","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434442+0000","last_change":"2024-09-10T18:53:24.289383+0000","last_active":"2024-09-10T18:53:37.434442+0000","last_peered":"2024-09-10T18:53:37.434442+0000","last_clean":"2024-09-10T18:53:37.434442+0000","last_became_active":"2024-09-10T18:53:24.289122+0000","last_became_peered":"2024-09-10T18:53:24.289122+0000","last_unstale":"2024-09-10T18:53:37.434442+0000","last_undegraded":"2024-09-10T18:53:37.434442+0000","last_fullsized":"2024-09-10T18:53:37.434442+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:52:59.862994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,0],"acting":[2,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435546+0000","last_change":"2024-09-10T18:53:28.304502+0000","last_active":"2024-09-10T18:53:37.435546+0000","last_peered":"2024-09-10T18:53:37.435546+0000","last_clean":"2024-09-10T18:53:37.435546+0000","last_became_active":"2024-09-10T18:53:28.304254+0000","last_became_peered":"2024-09-10T18:53:28.304254+0000","last_unstale":"2024-09-10T18:53:37.435546+0000","last_undegraded":"2024-09-10T18:53:37.435546+0000","last_fullsized":"2024-09-10T18:53:37.435546+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:00:21.344115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435095+0000","last_change":"2024-09-10T18:53:30.317564+0000","last_active":"2024-09-10T18:53:37.435095+0000","last_peered":"2024-09-10T18:53:37.435095+0000","last_clean":"2024-09-10T18:53:37.435095+0000","last_became_active":"2024-09-10T18:53:30.317271+0000","last_became_peered":"2024-09-10T18:53:30.317271+0000","last_unstale":"2024-09-10T18:53:37.435095+0000","last_undegraded":"2024-09-10T18:53:37.435095+0000","last_fullsized":"2024-09-10T18:53:37.435095+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:13:18.758778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4,7],"acting":[2,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.11","version":"52'1","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363121+0000","last_change":"2024-09-10T18:53:24.315004+0000","last_active":"2024-09-10T18:53:37.363121+0000","last_peered":"2024-09-10T18:53:37.363121+0000","last_clean":"2024-09-10T18:53:37.363121+0000","last_became_active":"2024-09-10T18:53:24.314611+0000","last_became_peered":"2024-09-10T18:53:24.314611+0000","last_unstale":"2024-09-10T18:53:37.363121+0000","last_undegraded":"2024-09-10T18:53:37.363121+0000","last_fullsized":"2024-09-10T18:53:37.363121+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:22:43.866314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":362,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,1],"acting":[6,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"59'6","reported_seq":25,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435570+0000","last_change":"2024-09-10T18:53:26.436607+0000","last_active":"2024-09-10T18:53:37.435570+0000","last_peered":"2024-09-10T18:53:37.435570+0000","last_clean":"2024-09-10T18:53:37.435570+0000","last_became_active":"2024-09-10T18:53:26.436341+0000","last_became_peered":"2024-09-10T18:53:26.436341+0000","last_unstale":"2024-09-10T18:53:37.435570+0000","last_undegraded":"2024-09-10T18:53:37.435570+0000","last_fullsized":"2024-09-10T18:53:37.435570+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:22:17.058998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":9,"num_read_kb":6,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1],"acting":[3,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435299+0000","last_change":"2024-09-10T18:53:28.303261+0000","last_active":"2024-09-10T18:53:37.435299+0000","last_peered":"2024-09-10T18:53:37.435299+0000","last_clean":"2024-09-10T18:53:37.435299+0000","last_became_active":"2024-09-10T18:53:28.302954+0000","last_became_peered":"2024-09-10T18:53:28.302954+0000","last_unstale":"2024-09-10T18:53:37.435299+0000","last_undegraded":"2024-09-10T18:53:37.435299+0000","last_fullsized":"2024-09-10T18:53:37.435299+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:04:16.935788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,1],"acting":[5,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434988+0000","last_change":"2024-09-10T18:53:30.494848+0000","last_active":"2024-09-10T18:53:37.434988+0000","last_peered":"2024-09-10T18:53:37.434988+0000","last_clean":"2024-09-10T18:53:37.434988+0000","last_became_active":"2024-09-10T18:53:30.494590+0000","last_became_peered":"2024-09-10T18:53:30.494590+0000","last_unstale":"2024-09-10T18:53:37.434988+0000","last_undegraded":"2024-09-10T18:53:37.434988+0000","last_fullsized":"2024-09-10T18:53:37.434988+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:42:16.904105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,4],"acting":[7,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435118+0000","last_change":"2024-09-10T18:53:26.434347+0000","last_active":"2024-09-10T18:53:37.435118+0000","last_peered":"2024-09-10T18:53:37.435118+0000","last_clean":"2024-09-10T18:53:37.435118+0000","last_became_active":"2024-09-10T18:53:26.434069+0000","last_became_peered":"2024-09-10T18:53:26.434069+0000","last_unstale":"2024-09-10T18:53:37.435118+0000","last_undegraded":"2024-09-10T18:53:37.435118+0000","last_fullsized":"2024-09-10T18:53:37.435118+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:30:08.159776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,7],"acting":[3,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434802+0000","last_change":"2024-09-10T18:53:24.315075+0000","last_active":"2024-09-10T18:53:37.434802+0000","last_peered":"2024-09-10T18:53:37.434802+0000","last_clean":"2024-09-10T18:53:37.434802+0000","last_became_active":"2024-09-10T18:53:24.314551+0000","last_became_peered":"2024-09-10T18:53:24.314551+0000","last_unstale":"2024-09-10T18:53:37.434802+0000","last_undegraded":"2024-09-10T18:53:37.434802+0000","last_fullsized":"2024-09-10T18:53:37.434802+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:58:46.067240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543099+0000","last_change":"2024-09-10T18:53:28.302415+0000","last_active":"2024-09-10T18:54:37.543099+0000","last_peered":"2024-09-10T18:54:37.543099+0000","last_clean":"2024-09-10T18:54:37.543099+0000","last_became_active":"2024-09-10T18:53:28.302125+0000","last_became_peered":"2024-09-10T18:53:28.302125+0000","last_unstale":"2024-09-10T18:54:37.543099+0000","last_undegraded":"2024-09-10T18:54:37.543099+0000","last_fullsized":"2024-09-10T18:54:37.543099+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:26:51.115976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,0],"acting":[5,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151073+0000","last_change":"2024-09-10T18:53:30.318114+0000","last_active":"2024-09-10T18:53:37.151073+0000","last_peered":"2024-09-10T18:53:37.151073+0000","last_clean":"2024-09-10T18:53:37.151073+0000","last_became_active":"2024-09-10T18:53:30.317816+0000","last_became_peered":"2024-09-10T18:53:30.317816+0000","last_unstale":"2024-09-10T18:53:37.151073+0000","last_undegraded":"2024-09-10T18:53:37.151073+0000","last_fullsized":"2024-09-10T18:53:37.151073+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:56:55.124939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434564+0000","last_change":"2024-09-10T18:53:26.434624+0000","last_active":"2024-09-10T18:53:37.434564+0000","last_peered":"2024-09-10T18:53:37.434564+0000","last_clean":"2024-09-10T18:53:37.434564+0000","last_became_active":"2024-09-10T18:53:26.434286+0000","last_became_peered":"2024-09-10T18:53:26.434286+0000","last_unstale":"2024-09-10T18:53:37.434564+0000","last_undegraded":"2024-09-10T18:53:37.434564+0000","last_fullsized":"2024-09-10T18:53:37.434564+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:39:17.113298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7,3],"acting":[5,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151511+0000","last_change":"2024-09-10T18:53:24.289506+0000","last_active":"2024-09-10T18:53:37.151511+0000","last_peered":"2024-09-10T18:53:37.151511+0000","last_clean":"2024-09-10T18:53:37.151511+0000","last_became_active":"2024-09-10T18:53:24.289093+0000","last_became_peered":"2024-09-10T18:53:24.289093+0000","last_unstale":"2024-09-10T18:53:37.151511+0000","last_undegraded":"2024-09-10T18:53:37.151511+0000","last_fullsized":"2024-09-10T18:53:37.151511+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:29:53.254037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,3],"acting":[0,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"59'8","reported_seq":38,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.542752+0000","last_change":"2024-09-10T18:53:28.306676+0000","last_active":"2024-09-10T18:54:37.542752+0000","last_peered":"2024-09-10T18:54:37.542752+0000","last_clean":"2024-09-10T18:54:37.542752+0000","last_became_active":"2024-09-10T18:53:28.306371+0000","last_became_peered":"2024-09-10T18:53:28.306371+0000","last_unstale":"2024-09-10T18:54:37.542752+0000","last_undegraded":"2024-09-10T18:54:37.542752+0000","last_fullsized":"2024-09-10T18:54:37.542752+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:02:16.511464+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,2],"acting":[3,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148762+0000","last_change":"2024-09-10T18:53:30.319982+0000","last_active":"2024-09-10T18:53:37.148762+0000","last_peered":"2024-09-10T18:53:37.148762+0000","last_clean":"2024-09-10T18:53:37.148762+0000","last_became_active":"2024-09-10T18:53:30.319386+0000","last_became_peered":"2024-09-10T18:53:30.319386+0000","last_unstale":"2024-09-10T18:53:37.148762+0000","last_undegraded":"2024-09-10T18:53:37.148762+0000","last_fullsized":"2024-09-10T18:53:37.148762+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:53:42.892780+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,5],"acting":[4,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.12","version":"59'9","reported_seq":32,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150492+0000","last_change":"2024-09-10T18:53:26.301906+0000","last_active":"2024-09-10T18:53:37.150492+0000","last_peered":"2024-09-10T18:53:37.150492+0000","last_clean":"2024-09-10T18:53:37.150492+0000","last_became_active":"2024-09-10T18:53:26.301613+0000","last_became_peered":"2024-09-10T18:53:26.301613+0000","last_unstale":"2024-09-10T18:53:37.150492+0000","last_undegraded":"2024-09-10T18:53:37.150492+0000","last_fullsized":"2024-09-10T18:53:37.150492+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:03:28.086658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":17,"num_read_kb":11,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6,2],"acting":[1,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363259+0000","last_change":"2024-09-10T18:53:24.317867+0000","last_active":"2024-09-10T18:53:37.363259+0000","last_peered":"2024-09-10T18:53:37.363259+0000","last_clean":"2024-09-10T18:53:37.363259+0000","last_became_active":"2024-09-10T18:53:24.317382+0000","last_became_peered":"2024-09-10T18:53:24.317382+0000","last_unstale":"2024-09-10T18:53:37.363259+0000","last_undegraded":"2024-09-10T18:53:37.363259+0000","last_fullsized":"2024-09-10T18:53:37.363259+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:16:12.670061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,5],"acting":[6,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434557+0000","last_change":"2024-09-10T18:53:28.303638+0000","last_active":"2024-09-10T18:53:37.434557+0000","last_peered":"2024-09-10T18:53:37.434557+0000","last_clean":"2024-09-10T18:53:37.434557+0000","last_became_active":"2024-09-10T18:53:28.303301+0000","last_became_peered":"2024-09-10T18:53:28.303301+0000","last_unstale":"2024-09-10T18:53:37.434557+0000","last_undegraded":"2024-09-10T18:53:37.434557+0000","last_fullsized":"2024-09-10T18:53:37.434557+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:49:54.744610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,1],"acting":[3,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150732+0000","last_change":"2024-09-10T18:53:30.321476+0000","last_active":"2024-09-10T18:53:37.150732+0000","last_peered":"2024-09-10T18:53:37.150732+0000","last_clean":"2024-09-10T18:53:37.150732+0000","last_became_active":"2024-09-10T18:53:30.321183+0000","last_became_peered":"2024-09-10T18:53:30.321183+0000","last_unstale":"2024-09-10T18:53:37.150732+0000","last_undegraded":"2024-09-10T18:53:37.150732+0000","last_fullsized":"2024-09-10T18:53:37.150732+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:03:32.580166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5,1],"acting":[0,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148276+0000","last_change":"2024-09-10T18:53:26.297537+0000","last_active":"2024-09-10T18:53:37.148276+0000","last_peered":"2024-09-10T18:53:37.148276+0000","last_clean":"2024-09-10T18:53:37.148276+0000","last_became_active":"2024-09-10T18:53:26.297170+0000","last_became_peered":"2024-09-10T18:53:26.297170+0000","last_unstale":"2024-09-10T18:53:37.148276+0000","last_undegraded":"2024-09-10T18:53:37.148276+0000","last_fullsized":"2024-09-10T18:53:37.148276+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:36:42.333587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,1],"acting":[4,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150489+0000","last_change":"2024-09-10T18:53:24.311844+0000","last_active":"2024-09-10T18:53:37.150489+0000","last_peered":"2024-09-10T18:53:37.150489+0000","last_clean":"2024-09-10T18:53:37.150489+0000","last_became_active":"2024-09-10T18:53:24.311613+0000","last_became_peered":"2024-09-10T18:53:24.311613+0000","last_unstale":"2024-09-10T18:53:37.150489+0000","last_undegraded":"2024-09-10T18:53:37.150489+0000","last_fullsized":"2024-09-10T18:53:37.150489+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:22:18.967074+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,0],"acting":[1,5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150402+0000","last_change":"2024-09-10T18:53:28.310289+0000","last_active":"2024-09-10T18:53:37.150402+0000","last_peered":"2024-09-10T18:53:37.150402+0000","last_clean":"2024-09-10T18:53:37.150402+0000","last_became_active":"2024-09-10T18:53:28.310005+0000","last_became_peered":"2024-09-10T18:53:28.310005+0000","last_unstale":"2024-09-10T18:53:37.150402+0000","last_undegraded":"2024-09-10T18:53:37.150402+0000","last_fullsized":"2024-09-10T18:53:37.150402+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:25:54.223793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,3],"acting":[1,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434162+0000","last_change":"2024-09-10T18:53:30.315912+0000","last_active":"2024-09-10T18:53:37.434162+0000","last_peered":"2024-09-10T18:53:37.434162+0000","last_clean":"2024-09-10T18:53:37.434162+0000","last_became_active":"2024-09-10T18:53:30.315634+0000","last_became_peered":"2024-09-10T18:53:30.315634+0000","last_unstale":"2024-09-10T18:53:37.434162+0000","last_undegraded":"2024-09-10T18:53:37.434162+0000","last_fullsized":"2024-09-10T18:53:37.434162+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:45.677581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"59'4","reported_seq":22,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434172+0000","last_change":"2024-09-10T18:53:26.297801+0000","last_active":"2024-09-10T18:53:37.434172+0000","last_peered":"2024-09-10T18:53:37.434172+0000","last_clean":"2024-09-10T18:53:37.434172+0000","last_became_active":"2024-09-10T18:53:26.297441+0000","last_became_peered":"2024-09-10T18:53:26.297441+0000","last_unstale":"2024-09-10T18:53:37.434172+0000","last_undegraded":"2024-09-10T18:53:37.434172+0000","last_fullsized":"2024-09-10T18:53:37.434172+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:41:49.951831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,6],"acting":[3,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435414+0000","last_change":"2024-09-10T18:53:24.315871+0000","last_active":"2024-09-10T18:53:37.435414+0000","last_peered":"2024-09-10T18:53:37.435414+0000","last_clean":"2024-09-10T18:53:37.435414+0000","last_became_active":"2024-09-10T18:53:24.315514+0000","last_became_peered":"2024-09-10T18:53:24.315514+0000","last_unstale":"2024-09-10T18:53:37.435414+0000","last_undegraded":"2024-09-10T18:53:37.435414+0000","last_fullsized":"2024-09-10T18:53:37.435414+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:42:05.010080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6,2],"acting":[5,6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.362938+0000","last_change":"2024-09-10T18:53:28.300959+0000","last_active":"2024-09-10T18:53:37.362938+0000","last_peered":"2024-09-10T18:53:37.362938+0000","last_clean":"2024-09-10T18:53:37.362938+0000","last_became_active":"2024-09-10T18:53:28.300657+0000","last_became_peered":"2024-09-10T18:53:28.300657+0000","last_unstale":"2024-09-10T18:53:37.362938+0000","last_undegraded":"2024-09-10T18:53:37.362938+0000","last_fullsized":"2024-09-10T18:53:37.362938+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:14:48.938593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434702+0000","last_change":"2024-09-10T18:53:30.318413+0000","last_active":"2024-09-10T18:53:37.434702+0000","last_peered":"2024-09-10T18:53:37.434702+0000","last_clean":"2024-09-10T18:53:37.434702+0000","last_became_active":"2024-09-10T18:53:30.318138+0000","last_became_peered":"2024-09-10T18:53:30.318138+0000","last_unstale":"2024-09-10T18:53:37.434702+0000","last_undegraded":"2024-09-10T18:53:37.434702+0000","last_fullsized":"2024-09-10T18:53:37.434702+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:56:16.909131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2,4],"acting":[7,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434249+0000","last_change":"2024-09-10T18:53:26.434338+0000","last_active":"2024-09-10T18:53:37.434249+0000","last_peered":"2024-09-10T18:53:37.434249+0000","last_clean":"2024-09-10T18:53:37.434249+0000","last_became_active":"2024-09-10T18:53:26.434083+0000","last_became_peered":"2024-09-10T18:53:26.434083+0000","last_unstale":"2024-09-10T18:53:37.434249+0000","last_undegraded":"2024-09-10T18:53:37.434249+0000","last_fullsized":"2024-09-10T18:53:37.434249+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T23:11:10.922765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363510+0000","last_change":"2024-09-10T18:53:24.317730+0000","last_active":"2024-09-10T18:53:37.363510+0000","last_peered":"2024-09-10T18:53:37.363510+0000","last_clean":"2024-09-10T18:53:37.363510+0000","last_became_active":"2024-09-10T18:53:24.317363+0000","last_became_peered":"2024-09-10T18:53:24.317363+0000","last_unstale":"2024-09-10T18:53:37.363510+0000","last_undegraded":"2024-09-10T18:53:37.363510+0000","last_fullsized":"2024-09-10T18:53:37.363510+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:40.286267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,2],"acting":[6,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434251+0000","last_change":"2024-09-10T18:53:28.299121+0000","last_active":"2024-09-10T18:53:37.434251+0000","last_peered":"2024-09-10T18:53:37.434251+0000","last_clean":"2024-09-10T18:53:37.434251+0000","last_became_active":"2024-09-10T18:53:28.298766+0000","last_became_peered":"2024-09-10T18:53:28.298766+0000","last_unstale":"2024-09-10T18:53:37.434251+0000","last_undegraded":"2024-09-10T18:53:37.434251+0000","last_fullsized":"2024-09-10T18:53:37.434251+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:40:47.151320+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434246+0000","last_change":"2024-09-10T18:53:30.495927+0000","last_active":"2024-09-10T18:53:37.434246+0000","last_peered":"2024-09-10T18:53:37.434246+0000","last_clean":"2024-09-10T18:53:37.434246+0000","last_became_active":"2024-09-10T18:53:30.495510+0000","last_became_peered":"2024-09-10T18:53:30.495510+0000","last_unstale":"2024-09-10T18:53:37.434246+0000","last_undegraded":"2024-09-10T18:53:37.434246+0000","last_fullsized":"2024-09-10T18:53:37.434246+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:49:42.354276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,6],"acting":[3,0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1c","version":"59'1","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435463+0000","last_change":"2024-09-10T18:53:30.318875+0000","last_active":"2024-09-10T18:53:37.435463+0000","last_peered":"2024-09-10T18:53:37.435463+0000","last_clean":"2024-09-10T18:53:37.435463+0000","last_became_active":"2024-09-10T18:53:30.318611+0000","last_became_peered":"2024-09-10T18:53:30.318611+0000","last_unstale":"2024-09-10T18:53:37.435463+0000","last_undegraded":"2024-09-10T18:53:37.435463+0000","last_fullsized":"2024-09-10T18:53:37.435463+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:54:22.747007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":369,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,2],"acting":[7,5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434632+0000","last_change":"2024-09-10T18:53:24.317163+0000","last_active":"2024-09-10T18:53:37.434632+0000","last_peered":"2024-09-10T18:53:37.434632+0000","last_clean":"2024-09-10T18:53:37.434632+0000","last_became_active":"2024-09-10T18:53:24.315764+0000","last_became_peered":"2024-09-10T18:53:24.315764+0000","last_unstale":"2024-09-10T18:53:37.434632+0000","last_undegraded":"2024-09-10T18:53:37.434632+0000","last_fullsized":"2024-09-10T18:53:37.434632+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:33:14.759149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,7],"acting":[5,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1e","version":"59'10","reported_seq":31,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151328+0000","last_change":"2024-09-10T18:53:26.437213+0000","last_active":"2024-09-10T18:53:37.151328+0000","last_peered":"2024-09-10T18:53:37.151328+0000","last_clean":"2024-09-10T18:53:37.151328+0000","last_became_active":"2024-09-10T18:53:26.436935+0000","last_became_peered":"2024-09-10T18:53:26.436935+0000","last_unstale":"2024-09-10T18:53:37.151328+0000","last_undegraded":"2024-09-10T18:53:37.151328+0000","last_fullsized":"2024-09-10T18:53:37.151328+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T20:31:04.828657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,3],"acting":[0,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1f","version":"59'8","reported_seq":44,"reported_epoch":61,"state":"active+clean","last_fresh":"2024-09-10T18:54:37.543102+0000","last_change":"2024-09-10T18:53:28.301965+0000","last_active":"2024-09-10T18:54:37.543102+0000","last_peered":"2024-09-10T18:54:37.543102+0000","last_clean":"2024-09-10T18:54:37.543102+0000","last_became_active":"2024-09-10T18:53:28.301640+0000","last_became_peered":"2024-09-10T18:53:28.301640+0000","last_unstale":"2024-09-10T18:54:37.543102+0000","last_undegraded":"2024-09-10T18:54:37.543102+0000","last_fullsized":"2024-09-10T18:54:37.543102+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:39:02.876161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150025+0000","last_change":"2024-09-10T18:53:30.317510+0000","last_active":"2024-09-10T18:53:37.150025+0000","last_peered":"2024-09-10T18:53:37.150025+0000","last_clean":"2024-09-10T18:53:37.150025+0000","last_became_active":"2024-09-10T18:53:30.317222+0000","last_became_peered":"2024-09-10T18:53:30.317222+0000","last_unstale":"2024-09-10T18:53:37.150025+0000","last_undegraded":"2024-09-10T18:53:37.150025+0000","last_fullsized":"2024-09-10T18:53:37.150025+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T05:34:31.429631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,4],"acting":[1,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434453+0000","last_change":"2024-09-10T18:53:24.315766+0000","last_active":"2024-09-10T18:53:37.434453+0000","last_peered":"2024-09-10T18:53:37.434453+0000","last_clean":"2024-09-10T18:53:37.434453+0000","last_became_active":"2024-09-10T18:53:24.315411+0000","last_became_peered":"2024-09-10T18:53:24.315411+0000","last_unstale":"2024-09-10T18:53:37.434453+0000","last_undegraded":"2024-09-10T18:53:37.434453+0000","last_fullsized":"2024-09-10T18:53:37.434453+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:49:02.358653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,0],"acting":[3,6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"59'11","reported_seq":35,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363480+0000","last_change":"2024-09-10T18:53:26.299889+0000","last_active":"2024-09-10T18:53:37.363480+0000","last_peered":"2024-09-10T18:53:37.363480+0000","last_clean":"2024-09-10T18:53:37.363480+0000","last_became_active":"2024-09-10T18:53:26.299584+0000","last_became_peered":"2024-09-10T18:53:26.299584+0000","last_unstale":"2024-09-10T18:53:37.363480+0000","last_undegraded":"2024-09-10T18:53:37.363480+0000","last_fullsized":"2024-09-10T18:53:37.363480+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:44:39.636331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":20,"num_read_kb":13,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,1],"acting":[6,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.151236+0000","last_change":"2024-09-10T18:53:28.309275+0000","last_active":"2024-09-10T18:53:37.151236+0000","last_peered":"2024-09-10T18:53:37.151236+0000","last_clean":"2024-09-10T18:53:37.151236+0000","last_became_active":"2024-09-10T18:53:28.308803+0000","last_became_peered":"2024-09-10T18:53:28.308803+0000","last_unstale":"2024-09-10T18:53:37.151236+0000","last_undegraded":"2024-09-10T18:53:37.151236+0000","last_fullsized":"2024-09-10T18:53:37.151236+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T06:10:50.786087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,2],"acting":[0,7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148652+0000","last_change":"2024-09-10T18:53:30.493741+0000","last_active":"2024-09-10T18:53:37.148652+0000","last_peered":"2024-09-10T18:53:37.148652+0000","last_clean":"2024-09-10T18:53:37.148652+0000","last_became_active":"2024-09-10T18:53:30.493445+0000","last_became_peered":"2024-09-10T18:53:30.493445+0000","last_unstale":"2024-09-10T18:53:37.148652+0000","last_undegraded":"2024-09-10T18:53:37.148652+0000","last_fullsized":"2024-09-10T18:53:37.148652+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T03:27:02.419396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5],"acting":[4,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1a","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.363394+0000","last_change":"2024-09-10T18:53:24.316691+0000","last_active":"2024-09-10T18:53:37.363394+0000","last_peered":"2024-09-10T18:53:37.363394+0000","last_clean":"2024-09-10T18:53:37.363394+0000","last_became_active":"2024-09-10T18:53:24.316415+0000","last_became_peered":"2024-09-10T18:53:24.316415+0000","last_unstale":"2024-09-10T18:53:37.363394+0000","last_undegraded":"2024-09-10T18:53:37.363394+0000","last_fullsized":"2024-09-10T18:53:37.363394+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T01:31:21.215771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,7],"acting":[6,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1c","version":"59'15","reported_seq":41,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435351+0000","last_change":"2024-09-10T18:53:26.303843+0000","last_active":"2024-09-10T18:53:37.435351+0000","last_peered":"2024-09-10T18:53:37.435351+0000","last_clean":"2024-09-10T18:53:37.435351+0000","last_became_active":"2024-09-10T18:53:26.303472+0000","last_became_peered":"2024-09-10T18:53:26.303472+0000","last_unstale":"2024-09-10T18:53:37.435351+0000","last_undegraded":"2024-09-10T18:53:37.435351+0000","last_fullsized":"2024-09-10T18:53:37.435351+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T22:56:00.330500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":26,"num_read_kb":17,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,3],"acting":[2,1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.150180+0000","last_change":"2024-09-10T18:53:28.302606+0000","last_active":"2024-09-10T18:53:37.150180+0000","last_peered":"2024-09-10T18:53:37.150180+0000","last_clean":"2024-09-10T18:53:37.150180+0000","last_became_active":"2024-09-10T18:53:28.302313+0000","last_became_peered":"2024-09-10T18:53:28.302313+0000","last_unstale":"2024-09-10T18:53:37.150180+0000","last_undegraded":"2024-09-10T18:53:37.150180+0000","last_fullsized":"2024-09-10T18:53:37.150180+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T21:55:21.327513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4,0],"acting":[1,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435483+0000","last_change":"2024-09-10T18:53:30.493777+0000","last_active":"2024-09-10T18:53:37.435483+0000","last_peered":"2024-09-10T18:53:37.435483+0000","last_clean":"2024-09-10T18:53:37.435483+0000","last_became_active":"2024-09-10T18:53:30.493491+0000","last_became_peered":"2024-09-10T18:53:30.493491+0000","last_unstale":"2024-09-10T18:53:37.435483+0000","last_undegraded":"2024-09-10T18:53:37.435483+0000","last_fullsized":"2024-09-10T18:53:37.435483+0000","mapping_epoch":57,"log_start":"0'0","ondisk_log_start":"0'0","created":57,"last_epoch_clean":58,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:29.294722+0000","last_clean_scrub_stamp":"2024-09-10T18:53:29.294722+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T00:40:54.114679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6,5],"acting":[3,6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1b","version":"0'0","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.434649+0000","last_change":"2024-09-10T18:53:24.321515+0000","last_active":"2024-09-10T18:53:37.434649+0000","last_peered":"2024-09-10T18:53:37.434649+0000","last_clean":"2024-09-10T18:53:37.434649+0000","last_became_active":"2024-09-10T18:53:24.321197+0000","last_became_peered":"2024-09-10T18:53:24.321197+0000","last_unstale":"2024-09-10T18:53:37.434649+0000","last_undegraded":"2024-09-10T18:53:37.434649+0000","last_fullsized":"2024-09-10T18:53:37.434649+0000","mapping_epoch":51,"log_start":"0'0","ondisk_log_start":"0'0","created":51,"last_epoch_clean":52,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:23.268570+0000","last_clean_scrub_stamp":"2024-09-10T18:53:23.268570+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T02:52:56.925730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,6],"acting":[3,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"59'12","reported_seq":39,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.435390+0000","last_change":"2024-09-10T18:53:26.291345+0000","last_active":"2024-09-10T18:53:37.435390+0000","last_peered":"2024-09-10T18:53:37.435390+0000","last_clean":"2024-09-10T18:53:37.435390+0000","last_became_active":"2024-09-10T18:53:26.291067+0000","last_became_peered":"2024-09-10T18:53:26.291067+0000","last_unstale":"2024-09-10T18:53:37.435390+0000","last_undegraded":"2024-09-10T18:53:37.435390+0000","last_fullsized":"2024-09-10T18:53:37.435390+0000","mapping_epoch":53,"log_start":"0'0","ondisk_log_start":"0'0","created":53,"last_epoch_clean":54,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:25.276530+0000","last_clean_scrub_stamp":"2024-09-10T18:53:25.276530+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-12T04:27:28.293789+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":25,"num_read_kb":16,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1,4],"acting":[3,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":14,"reported_epoch":60,"state":"active+clean","last_fresh":"2024-09-10T18:53:37.148492+0000","last_change":"2024-09-10T18:53:28.306652+0000","last_active":"2024-09-10T18:53:37.148492+0000","last_peered":"2024-09-10T18:53:37.148492+0000","last_clean":"2024-09-10T18:53:37.148492+0000","last_became_active":"2024-09-10T18:53:28.306329+0000","last_became_peered":"2024-09-10T18:53:28.306329+0000","last_unstale":"2024-09-10T18:53:37.148492+0000","last_undegraded":"2024-09-10T18:53:37.148492+0000","last_fullsized":"2024-09-10T18:53:37.148492+0000","mapping_epoch":55,"log_start":"0'0","ondisk_log_start":"0'0","created":55,"last_epoch_clean":56,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-10T18:53:27.282079+0000","last_clean_scrub_stamp":"2024-09-10T18:53:27.282079+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-11T19:14:55.731981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,2],"acting":[4,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":382,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":96,"acting":96,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":64,"ondisk_log_size":64,"up":96,"acting":96,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":3702,"num_objects":177,"num_object_clones":0,"num_object_copies":531,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":177,"num_whiteouts":0,"num_read":698,"num_read_kb":455,"num_write":417,"num_write_kb":34,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":417792,"data_stored":11106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":391,"ondisk_log_size":391,"up":96,"acting":96,"num_store_stats":8},{"poolid":3,"num_pg":3,"stat_sum":{"num_bytes":408,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":71,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1224,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":8,"ondisk_log_size":8,"up":9,"acting":9,"num_store_stats":7},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":1328,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4,"ondisk_log_size":4,"up":96,"acting":96,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":231,"num_write":113,"num_write_kb":1628,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":590368,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":79,"ondisk_log_size":79,"up":3,"acting":3,"num_store_stats":3}],"osd_stats":[{"osd":7,"up_from":48,"seq":206158430230,"num_pgs":57,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1240,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1269760,"data_stored":868553,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":1.2090000000000001,"5min":1.2090000000000001,"15min":1.2090000000000001},"last":0.46600000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":1.173,"5min":1.173,"15min":1.173},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.49399999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.52900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.1619999999999999,"5min":1.1619999999999999,"15min":1.1619999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.53200000000000003},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.76700000000000002,"5min":0.76700000000000002,"15min":0.76700000000000002},"last":0.42599999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.63400000000000001},"last":0.40799999999999997}]},{"osd":5,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"last":0.38200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.29999999999999999,"5min":0.29999999999999999,"15min":0.29999999999999999},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.59899999999999998}]}]},{"osd":6,"up_from":42,"seq":180388626458,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298572,"kb_used_data":1220,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446580,"statfs":{"total":95995035648,"available":95689297920,"internally_reserved":0,"allocated":1249280,"data_stored":867138,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.253,"5min":0.253,"15min":0.253},"max":{"1min":1.3580000000000001,"5min":1.3580000000000001,"15min":1.3580000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.53800000000000003}]},{"osd":1,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.52900000000000003}]},{"osd":2,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.56799999999999995,"15min":0.56799999999999995},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.4399999999999999,"5min":1.4399999999999999,"15min":1.4399999999999999},"last":0.49399999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.048,"5min":1.048,"15min":1.048},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.28299999999999997,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.99299999999999999,"5min":0.99299999999999999,"15min":0.99299999999999999},"last":0.50900000000000001}]},{"osd":4,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.23699999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.437,"5min":0.437,"15min":0.437},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.79000000000000004,"5min":0.79000000000000004,"15min":0.79000000000000004},"last":0.39300000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"min":{"1min":0.27200000000000002,"5min":0.27200000000000002,"15min":0.27200000000000002},"max":{"1min":0.67200000000000004,"5min":0.67200000000000004,"15min":0.67200000000000004},"last":0.443},{"interface":"front","average":{"1min":0.441,"5min":0.441,"15min":0.441},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.623}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.499,"5min":0.499,"15min":0.499},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42799999999999999}]}]},{"osd":5,"up_from":36,"seq":154618822688,"num_pgs":48,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298004,"kb_used_data":648,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447148,"statfs":{"total":95995035648,"available":95689879552,"internally_reserved":0,"allocated":663552,"data_stored":276987,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.54800000000000004,"15min":0.55000000000000004},"min":{"1min":0.39900000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.65400000000000003,"5min":1.387,"15min":1.387},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.52800000000000002,"5min":0.54400000000000004,"15min":0.54600000000000004},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.69199999999999995,"5min":1.5169999999999999,"15min":1.5169999999999999},"last":0.40899999999999997}]},{"osd":1,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.57899999999999996,"15min":0.58499999999999996},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.71699999999999997,"5min":1.496,"15min":1.496},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.53800000000000003,"15min":0.53900000000000003},"min":{"1min":0.38700000000000001,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.69799999999999995,"5min":1.3120000000000001,"15min":1.3120000000000001},"last":0.45500000000000002}]},{"osd":2,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.54000000000000004,"5min":0.55000000000000004,"15min":0.55200000000000005},"min":{"1min":0.42799999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.63100000000000001,"5min":1.3999999999999999,"15min":1.3999999999999999},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.58299999999999996,"15min":0.58499999999999996},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.69299999999999995,"5min":1.534,"15min":1.534},"last":0.60999999999999999}]},{"osd":3,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.443,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.752,"5min":1.427,"15min":1.427},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.57099999999999995,"15min":0.56899999999999995},"min":{"1min":0.41999999999999998,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.74099999999999999,"5min":1.3260000000000001,"15min":1.3260000000000001},"last":0.68100000000000005}]},{"osd":4,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.42599999999999999,"5min":0.36299999999999999,"15min":0.35299999999999998},"min":{"1min":0.313,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.67500000000000004,"5min":0.67500000000000004,"15min":0.67500000000000004},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.45500000000000002,"5min":0.38100000000000001,"15min":0.36899999999999999},"min":{"1min":0.32400000000000001,"5min":0.25900000000000001,"15min":0.25900000000000001},"max":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.66300000000000003}]},{"osd":6,"last update":"Tue Sep 10 18:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67600000000000005,"5min":0.67600000000000005,"15min":0.67600000000000005},"last":0.59299999999999997}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.629},{"interface":"front","average":{"1min":0.503,"5min":0.503,"15min":0.503},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.71099999999999997}]}]},{"osd":4,"up_from":30,"seq":128849018916,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298020,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447132,"statfs":{"total":95995035648,"available":95689863168,"internally_reserved":0,"allocated":679936,"data_stored":277524,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.5,"5min":0.51200000000000001,"15min":0.51400000000000001},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.68200000000000005,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.53200000000000003,"15min":0.53300000000000003},"min":{"1min":0.38600000000000001,"5min":0.37,"15min":0.37},"max":{"1min":0.71999999999999997,"5min":0.96999999999999997,"15min":0.96999999999999997},"last":0.39500000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.54900000000000004,"15min":0.55700000000000005},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.872,"5min":1.052,"15min":1.052},"last":0.432},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.47999999999999998,"15min":0.47799999999999998},"min":{"1min":0.34399999999999997,"5min":0.34100000000000003,"15min":0.34100000000000003},"max":{"1min":0.63800000000000001,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.498}]},{"osd":2,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.50900000000000001,"15min":0.51000000000000001},"min":{"1min":0.311,"5min":0.311,"15min":0.311},"max":{"1min":0.66300000000000003,"5min":0.84099999999999997,"15min":0.84099999999999997},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.5,"5min":0.54500000000000004,"15min":0.55300000000000005},"min":{"1min":0.36499999999999999,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.76400000000000001,"5min":0.98999999999999999,"15min":0.98999999999999999},"last":0.443}]},{"osd":3,"last update":"Tue Sep 10 18:54:03 2024","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.35699999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.67500000000000004,"5min":0.89300000000000002,"15min":0.89300000000000002},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.54300000000000004,"15min":0.54500000000000004},"min":{"1min":0.35599999999999998,"5min":0.35599999999999998,"15min":0.35599999999999998},"max":{"1min":0.65700000000000003,"5min":0.91900000000000004,"15min":0.91900000000000004},"last":0.45300000000000001}]},{"osd":5,"last update":"Tue Sep 10 18:54:26 2024","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.36599999999999999,"15min":0.35499999999999998},"min":{"1min":0.28100000000000003,"5min":0.247,"15min":0.247},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.30299999999999999},{"interface":"front","average":{"1min":0.42699999999999999,"5min":0.36799999999999999,"15min":0.35799999999999998},"min":{"1min":0.30399999999999999,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.65000000000000002,"5min":0.65000000000000002,"15min":0.65000000000000002},"last":0.34999999999999998}]},{"osd":6,"last update":"Tue Sep 10 18:53:48 2024","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":7.3010000000000002,"5min":7.3010000000000002,"15min":7.3010000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.498,"5min":0.498,"15min":0.498},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":2.516,"5min":2.516,"15min":2.516},"last":0.40400000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":2.9510000000000001,"5min":2.9510000000000001,"15min":2.9510000000000001},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":3.5449999999999999,"5min":3.5449999999999999,"15min":3.5449999999999999},"last":0.38400000000000001}]}]},{"osd":3,"up_from":25,"seq":107374182440,"num_pgs":55,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298040,"kb_used_data":684,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447112,"statfs":{"total":95995035648,"available":95689842688,"internally_reserved":0,"allocated":700416,"data_stored":278400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.34300000000000003,"5min":0.374,"15min":0.38100000000000001},"min":{"1min":0.20300000000000001,"5min":0.189,"15min":0.189},"max":{"1min":0.49299999999999999,"5min":0.51600000000000001,"15min":0.51600000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.32900000000000001,"5min":0.39600000000000002,"15min":0.41199999999999998},"min":{"1min":0.21299999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":0.501,"5min":0.57799999999999996,"15min":0.57799999999999996},"last":0.44}]},{"osd":1,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.35599999999999998,"5min":0.40300000000000002,"15min":0.41899999999999998},"min":{"1min":0.255,"5min":0.24199999999999999,"15min":0.24199999999999999},"max":{"1min":0.50800000000000001,"5min":0.52600000000000002,"15min":0.52600000000000002},"last":0.39700000000000002},{"interface":"front","average":{"1min":0.375,"5min":0.42699999999999999,"15min":0.434},"min":{"1min":0.24199999999999999,"5min":0.23699999999999999,"15min":0.23699999999999999},"max":{"1min":0.52300000000000002,"5min":0.65500000000000003,"15min":0.65500000000000003},"last":0.35599999999999998}]},{"osd":2,"last update":"Tue Sep 10 18:54:42 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.439,"15min":0.44900000000000001},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.628,"5min":0.628,"15min":0.628},"last":0.379},{"interface":"front","average":{"1min":0.34000000000000002,"5min":0.41499999999999998,"15min":0.434},"min":{"1min":0.254,"5min":0.254,"15min":0.254},"max":{"1min":0.42399999999999999,"5min":0.64600000000000002,"15min":0.64600000000000002},"last":0.34399999999999997}]},{"osd":4,"last update":"Tue Sep 10 18:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.56499999999999995,"5min":0.63700000000000001,"15min":0.64900000000000002},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.85399999999999998,"5min":1.9550000000000001,"15min":1.9550000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.55300000000000005,"5min":0.61499999999999999,"15min":0.625},"min":{"1min":0.35999999999999999,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.81200000000000006,"5min":1.4830000000000001,"15min":1.4830000000000001},"last":0.61599999999999999}]},{"osd":5,"last update":"Tue Sep 10 18:54:30 2024","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.60799999999999998,"15min":0.61699999999999999},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.81599999999999995,"5min":1.238,"15min":1.238},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.60899999999999999,"15min":0.61699999999999999},"min":{"1min":0.39400000000000002,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.69899999999999995,"5min":0.94999999999999996,"15min":0.94999999999999996},"last":0.68300000000000005}]},{"osd":6,"last update":"Tue Sep 10 18:53:52 2024","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.60099999999999998},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88900000000000001,"5min":0.88900000000000001,"15min":0.88900000000000001},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.59299999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.66600000000000004}]},{"osd":7,"last update":"Tue Sep 10 18:54:14 2024","interfaces":[{"interface":"back","average":{"1min":0.59699999999999998,"5min":0.59699999999999998,"15min":0.59699999999999998},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.877,"5min":0.877,"15min":0.877},"last":0.60799999999999998},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.86299999999999999,"5min":0.86299999999999999,"15min":0.86299999999999999},"last":0.65400000000000003}]}]},{"osd":2,"up_from":18,"seq":77309411373,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":297996,"kb_used_data":640,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447156,"statfs":{"total":95995035648,"available":95689887744,"internally_reserved":0,"allocated":655360,"data_stored":277152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.39000000000000001,"5min":0.36299999999999999,"15min":0.35399999999999998},"min":{"1min":0.223,"5min":0.223,"15min":0.223},"max":{"1min":0.92300000000000004,"5min":0.92300000000000004,"15min":0.92300000000000004},"last":0.29099999999999998},{"interface":"front","average":{"1min":0.371,"5min":0.36399999999999999,"15min":0.34699999999999998},"min":{"1min":0.23599999999999999,"5min":0.22,"15min":0.22},"max":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.61199999999999999},"last":0.40000000000000002}]},{"osd":1,"last update":"Tue Sep 10 18:54:16 2024","interfaces":[{"interface":"back","average":{"1min":0.38100000000000001,"5min":0.376,"15min":0.35699999999999998},"min":{"1min":0.247,"5min":0.247,"15min":0.247},"max":{"1min":0.95999999999999996,"5min":0.95999999999999996,"15min":0.95999999999999996},"last":0.318},{"interface":"front","average":{"1min":0.40200000000000002,"5min":0.38300000000000001,"15min":0.36699999999999999},"min":{"1min":0.249,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.35799999999999998}]},{"osd":3,"last update":"Tue Sep 10 18:54:41 2024","interfaces":[{"interface":"back","average":{"1min":0.38500000000000001,"5min":0.41599999999999998,"15min":0.42599999999999999},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.41799999999999998,"5min":0.442,"15min":0.45200000000000001},"min":{"1min":0.27300000000000002,"5min":0.27300000000000002,"15min":0.27300000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.378}]},{"osd":4,"last update":"Tue Sep 10 18:54:04 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.60899999999999999,"15min":0.61899999999999999},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.754,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.63200000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98999999999999999,"5min":1.0029999999999999,"15min":1.0029999999999999},"last":0.45500000000000002}]},{"osd":5,"last update":"Tue Sep 10 18:54:22 2024","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.60199999999999998,"15min":0.60599999999999998},"min":{"1min":0.39600000000000002,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.02,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.58999999999999997,"15min":0.59399999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.495}]},{"osd":6,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.628,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.2350000000000001,"5min":1.2350000000000001,"15min":1.2350000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.63,"5min":0.63,"15min":0.63},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":1.2490000000000001,"5min":1.2490000000000001,"15min":1.2490000000000001},"last":0.46100000000000002}]},{"osd":7,"last update":"Tue Sep 10 18:54:11 2024","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.60499999999999998,"15min":0.60499999999999998},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"last":0.39100000000000001},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.50600000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574898,"num_pgs":56,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298060,"kb_used_data":704,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93447092,"statfs":{"total":95995035648,"available":95689822208,"internally_reserved":0,"allocated":720896,"data_stored":279572,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.36699999999999999,"5min":0.33500000000000002,"15min":0.32700000000000001},"min":{"1min":0.214,"5min":0.214,"15min":0.214},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":1.0609999999999999},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.38900000000000001,"5min":0.35899999999999999,"15min":0.34399999999999997},"min":{"1min":0.23799999999999999,"5min":0.21299999999999999,"15min":0.21299999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.29699999999999999}]},{"osd":2,"last update":"Tue Sep 10 18:54:18 2024","interfaces":[{"interface":"back","average":{"1min":0.39100000000000001,"5min":0.40600000000000003,"15min":0.39900000000000002},"min":{"1min":0.28999999999999998,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":0.70699999999999996,"5min":0.97799999999999998,"15min":0.97799999999999998},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.36299999999999999,"15min":0.35799999999999998},"min":{"1min":0.248,"5min":0.215,"15min":0.215},"max":{"1min":0.52600000000000002,"5min":0.54700000000000004,"15min":0.54700000000000004},"last":0.26500000000000001}]},{"osd":3,"last update":"Tue Sep 10 18:54:40 2024","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.39300000000000002,"15min":0.40000000000000002},"min":{"1min":0.245,"5min":0.224,"15min":0.224},"max":{"1min":0.51100000000000001,"5min":0.95399999999999996,"15min":0.95399999999999996},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.372,"5min":0.41499999999999998,"15min":0.41699999999999998},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":0.48699999999999999,"5min":1.1799999999999999,"15min":1.1799999999999999},"last":0.39900000000000002}]},{"osd":4,"last update":"Tue Sep 10 18:54:00 2024","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.53000000000000003,"15min":0.52000000000000002},"min":{"1min":0.38,"5min":0.34399999999999997,"15min":0.34399999999999997},"max":{"1min":1.0880000000000001,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.57699999999999996,"15min":0.56599999999999995},"min":{"1min":0.41999999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":1.3939999999999999,"5min":1.3939999999999999,"15min":1.3939999999999999},"last":0.623}]},{"osd":5,"last update":"Tue Sep 10 18:54:28 2024","interfaces":[{"interface":"back","average":{"1min":0.51500000000000001,"5min":0.63200000000000001,"15min":0.65200000000000002},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72399999999999998,"5min":1.417,"15min":1.417},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.50900000000000001,"5min":0.58299999999999996,"15min":0.59499999999999997},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.72799999999999998,"5min":0.91800000000000004,"15min":0.91800000000000004},"last":0.61699999999999999}]},{"osd":6,"last update":"Tue Sep 10 18:53:47 2024","interfaces":[{"interface":"back","average":{"1min":0.63100000000000001,"5min":0.63100000000000001,"15min":0.63100000000000001},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.5389999999999999,"5min":1.5389999999999999,"15min":1.5389999999999999},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.53000000000000003}]},{"osd":7,"last update":"Tue Sep 10 18:54:12 2024","interfaces":[{"interface":"back","average":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.38800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.501},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.29,"5min":1.29,"15min":1.29},"last":0.64000000000000001}]}]},{"osd":0,"up_from":8,"seq":34359738424,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":298592,"kb_used_data":1236,"kb_used_omap":1,"kb_used_meta":297342,"kb_avail":93446560,"statfs":{"total":95995035648,"available":95689277440,"internally_reserved":0,"allocated":1265664,"data_stored":867376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1704,"internal_metadata":304478552},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Tue Sep 10 18:53:56 2024","interfaces":[{"interface":"back","average":{"1min":0.36499999999999999,"5min":0.32500000000000001,"15min":0.29399999999999998},"min":{"1min":0.26300000000000001,"5min":0.13900000000000001,"15min":0.13900000000000001},"max":{"1min":0.52200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"last":0.23300000000000001},{"interface":"front","average":{"1min":0.37,"5min":0.33100000000000002,"15min":0.29899999999999999},"min":{"1min":0.22700000000000001,"5min":0.17199999999999999,"15min":0.17199999999999999},"max":{"1min":0.54500000000000004,"5min":0.56000000000000005,"15min":0.56000000000000005},"last":0.26600000000000001}]},{"osd":2,"last update":"Tue Sep 10 18:54:21 2024","interfaces":[{"interface":"back","average":{"1min":0.38300000000000001,"5min":0.40000000000000002,"15min":0.39500000000000002},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.65200000000000002,"5min":0.65200000000000002,"15min":0.65200000000000002},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.36699999999999999,"15min":0.36099999999999999},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"last":0.252}]},{"osd":3,"last update":"Tue Sep 10 18:54:44 2024","interfaces":[{"interface":"back","average":{"1min":0.39500000000000002,"5min":0.44900000000000001,"15min":0.46300000000000002},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.61099999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"last":0.28100000000000003},{"interface":"front","average":{"1min":0.374,"5min":0.41999999999999998,"15min":0.433},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.46500000000000002,"5min":0.59199999999999997,"15min":0.59199999999999997},"last":0.30399999999999999}]},{"osd":4,"last update":"Tue Sep 10 18:54:01 2024","interfaces":[{"interface":"back","average":{"1min":0.52700000000000002,"5min":0.56699999999999995,"15min":0.57299999999999995},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.749,"5min":0.84299999999999997,"15min":0.84299999999999997},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.64900000000000002,"15min":0.66100000000000003},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77700000000000002,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.47299999999999998}]},{"osd":5,"last update":"Tue Sep 10 18:54:25 2024","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.82299999999999995,"15min":0.872},"min":{"1min":0.38,"5min":0.379,"15min":0.379},"max":{"1min":0.73499999999999999,"5min":7.8159999999999998,"15min":7.8159999999999998},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.56299999999999994,"5min":0.871,"15min":0.92200000000000004},"min":{"1min":0.37,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.80800000000000005,"5min":7.4249999999999998,"15min":7.4249999999999998},"last":0.51200000000000001}]},{"osd":6,"last update":"Tue Sep 10 18:53:51 2024","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85099999999999998,"5min":0.85099999999999998,"15min":0.85099999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.42699999999999999}]},{"osd":7,"last update":"Tue Sep 10 18:54:13 2024","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.78000000000000003,"5min":0.78000000000000003,"15min":0.78000000000000003},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.433}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":1236,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":874,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":454,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":362,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":966,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":389,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":19,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":1320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":2338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":73728,"data_stored":1898,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":53248,"data_stored":1474,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":990,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1034,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":13,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":369,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-10T18:54:52.761 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-10T18:54:52.761 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-10T18:54:52.761 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-10T18:54:52.761 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588 shell --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 -- ceph health --format=json 2024-09-10T18:54:53.005 INFO:teuthology.orchestra.run.smithi107.stderr:Inferring config /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/mon.a/config 2024-09-10T18:54:53.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:53 smithi107 ceph-mon[29404]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:53.372 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:53.372 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:53 smithi107 ceph-mon[40355]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:53.372 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:53.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:53 smithi162 ceph-mon[33228]: from='client.14700 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-10T18:54:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:54:54.404 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:54 smithi107 ceph-mon[29404]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:54.405 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:54 smithi107 ceph-mon[40355]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:54.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:54 smithi162 ceph-mon[33228]: pgmap v29: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 -- 172.21.15.107:0/638745832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 msgr2=0x7f7cc8101c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/638745832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc8101c00 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f7cb8004980 tx=0x7f7cb8009da0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 -- 172.21.15.107:0/638745832 shutdown_connections 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/638745832 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc81085e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/638745832 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc80fbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/638745832 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc8101c00 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 -- 172.21.15.107:0/638745832 >> 172.21.15.107:0/638745832 conn(0x7f7cc80713f0 msgr2=0x7f7cc80f99f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 -- 172.21.15.107:0/638745832 shutdown_connections 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.594+0000 7f7ccde0d640 1 -- 172.21.15.107:0/638745832 wait complete. 2024-09-10T18:54:54.596 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 Processor -- start 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 -- start start 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc8111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc8112130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc810c3d0 con 0x7f7cc80fc1c0 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc810c540 con 0x7f7cc80fb810 2024-09-10T18:54:54.598 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7ccde0d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cc810c700 con 0x7f7cc8101830 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc6ffd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc8112130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc8111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc7fff640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:33156/0 (socket says 172.21.15.107:33156) 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc7fff640 1 -- 172.21.15.107:0/1407595297 learned_addr learned my addr 172.21.15.107:0/1407595297 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 -- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 msgr2=0x7f7cc810bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 -- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 msgr2=0x7f7cc8112130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc8112130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc77fe640 1 -- 172.21.15.107:0/1407595297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cc00098b0 con 0x7f7cc80fb810 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc7fff640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.595+0000 7f7cc6ffd640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc8112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T18:54:54.599 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7cc77fe640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc8111bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7cb802a590 tx=0x7f7cb8035040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8007d00 con 0x7f7cc80fb810 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f7cb8033420 con 0x7f7cc80fb810 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cb8005040 con 0x7f7cc80fb810 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cb803c410 con 0x7f7cc80fb810 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.596+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cc8065f60 con 0x7f7cc80fb810 2024-09-10T18:54:54.600 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.597+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cc80fcf50 con 0x7f7cc80fb810 2024-09-10T18:54:54.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.597+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f7cb8030020 con 0x7f7cc80fb810 2024-09-10T18:54:54.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.598+0000 7f7cbffff640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 0x7f7cb40713c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T18:54:54.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.598+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 7417+0+0 (secure 0 0 0) 0x7f7cb80ae960 con 0x7f7cc80fb810 2024-09-10T18:54:54.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.598+0000 7f7cc6ffd640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 0x7f7cb40713c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T18:54:54.601 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.598+0000 7f7cc6ffd640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 0x7f7cb40713c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0004fd0 tx=0x7f7cb000b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T18:54:54.605 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.604+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f7cb807b4d0 con 0x7f7cc80fb810 2024-09-10T18:54:54.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.794+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f7cc810aaf0 con 0x7f7cc80fb810 2024-09-10T18:54:54.796 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.795+0000 7f7cbffff640 1 -- 172.21.15.107:0/1407595297 <== mon.1 v2:172.21.15.162:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f7cb807ae70 con 0x7f7cc80fb810 2024-09-10T18:54:54.796 INFO:teuthology.orchestra.run.smithi107.stdout: 2024-09-10T18:54:54.796 INFO:teuthology.orchestra.run.smithi107.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.796+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 msgr2=0x7f7cb40713c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.796+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 0x7f7cb40713c0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0004fd0 tx=0x7f7cb000b040 comp rx=0 tx=0).stop 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.796+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 msgr2=0x7f7cc8111bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.796+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc8111bf0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7cb802a590 tx=0x7f7cb8035040 comp rx=0 tx=0).stop 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 shutdown_connections 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f7cb406ef10 0x7f7cb40713c0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.798 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f7cc8101830 0x7f7cc810bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f7cc80fc1c0 0x7f7cc8112130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 --2- 172.21.15.107:0/1407595297 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f7cc80fb810 0x7f7cc8111bf0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T18:54:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 >> 172.21.15.107:0/1407595297 conn(0x7f7cc80713f0 msgr2=0x7f7cc8072970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T18:54:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 shutdown_connections 2024-09-10T18:54:54.799 INFO:teuthology.orchestra.run.smithi107.stderr:2024-09-10T18:54:54.797+0000 7f7ccde0d640 1 -- 172.21.15.107:0/1407595297 wait complete. 2024-09-10T18:54:55.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1407595297' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:54:55.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1407595297' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:54:55.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1407595297' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-09-10T18:54:55.523 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-10T18:54:55.524 INFO:tasks.cephadm:Setup complete, yielding 2024-09-10T18:54:55.524 INFO:teuthology.run_tasks:Running task workunit... 2024-09-10T18:54:55.539 INFO:tasks.workunit:Pulling workunits from ref fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:54:55.539 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-10T18:54:55.540 DEBUG:teuthology.orchestra.run.smithi107:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:54:55.556 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T18:54:55.556 INFO:teuthology.orchestra.run.smithi107.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2024-09-10T18:54:55.557 DEBUG:teuthology.orchestra.run.smithi107:> mkdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T18:54:55.612 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2024-09-10T18:54:55.613 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2024-09-10T18:54:55.667 INFO:tasks.workunit:timeout=3h 2024-09-10T18:54:55.667 INFO:tasks.workunit:cleanup=True 2024-09-10T18:54:55.668 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/adk3798/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout fad5356a94e6044ce1062d748daa5a0f2ed53a7d 2024-09-10T18:54:55.723 INFO:tasks.workunit.client.0.smithi107.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-10T18:54:56.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:56 smithi162 ceph-mon[33228]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:56 smithi107 ceph-mon[29404]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:56.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:56 smithi107 ceph-mon[40355]: pgmap v30: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:54:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:54:58 smithi162 ceph-mon[33228]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:54:58 smithi107 ceph-mon[29404]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:54:58 smithi107 ceph-mon[40355]: pgmap v31: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:54:59.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:54:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:54:59.022Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.003199437s 2024-09-10T18:55:00.372 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:00 smithi162 ceph-mon[33228]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:00 smithi107 ceph-mon[29404]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:00.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:00 smithi107 ceph-mon[40355]: pgmap v32: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:00.951 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:55:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:01.344 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:55:01.344 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.345 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-10T18:55:01.345 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.345 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:55:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-10T18:55:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]: dispatch 2024-09-10T18:55:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]: dispatch 2024-09-10T18:55:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]: dispatch 2024-09-10T18:55:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:01.651 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 podman[66856]: 2024-09-10 18:55:01.45348207 +0000 UTC m=+0.014945500 image pull 954c08fa618887fe9a6fe572ae541197c77e625b05f4193df7898f5ac649ada9 quay.io/ceph/ceph-grafana:9.4.7 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 podman[66856]: 2024-09-10 18:55:01.779807613 +0000 UTC m=+0.341271046 container init 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, vendor=Red Hat, Inc., distribution-scope=public, io.openshift.tags=base rhel8, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, architecture=x86_64, maintainer=Paul Cuzner , vcs-type=git, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., release=239.1651231664, com.redhat.component=ubi8-container, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, description=Ceph Grafana Container, version=8.5, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56) 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 podman[66856]: 2024-09-10 18:55:01.784523339 +0000 UTC m=+0.345986765 container start 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, com.redhat.component=ubi8-container, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, io.openshift.tags=base rhel8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=, summary=Grafana Container configured for Ceph mgr/dashboard integration, vendor=Red Hat, Inc., maintainer=Paul Cuzner , vcs-type=git, release=239.1651231664, io.buildah.version=1.32.0, io.k8s.display-name=Red Hat Universal Base Image 8, name=ubi8, architecture=x86_64) 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 bash[66856]: 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 systemd[1]: Started Ceph grafana.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943113366Z level=info msg="Starting Grafana" version=9.4.7 commit=4add91f03d branch=HEAD compiled=2023-03-16T23:56:52Z 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943330278Z level=warn msg="\"sentry\" frontend logging provider is deprecated and will be removed in the next major version. Use \"grafana\" provider instead." 2024-09-10T18:55:02.031 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943350798Z level=info msg="Config loaded from" file=/usr/share/grafana/conf/defaults.ini 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943359201Z level=info msg="Config loaded from" file=/etc/grafana/grafana.ini 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943366071Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_DATA=/var/lib/grafana" 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943372541Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_LOGS=/var/log/grafana" 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943379581Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PLUGINS=/var/lib/grafana/plugins" 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943386231Z level=info msg="Config overridden from Environment variable" var="GF_PATHS_PROVISIONING=/etc/grafana/provisioning" 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943393927Z level=info msg="Path Home" path=/usr/share/grafana 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943402217Z level=info msg="Path Data" path=/var/lib/grafana 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943408251Z level=info msg="Path Logs" path=/var/log/grafana 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943414026Z level=info msg="Path Plugins" path=/var/lib/grafana/plugins 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.943420138Z level=info msg="Path Provisioning" path=/etc/grafana/provisioning 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=settings t=2024-09-10T18:55:01.94342614Z level=info msg="App mode production" 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore t=2024-09-10T18:55:01.943480505Z level=info msg="Connecting to DB" dbtype=sqlite3 2024-09-10T18:55:02.032 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore t=2024-09-10T18:55:01.943505791Z level=warn msg="SQLite database file has broader permissions than it should" path=/var/lib/grafana/grafana.db mode=-rw-r--r-- expected=-rw-r----- 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.944354598Z level=info msg="Starting DB migrations" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.944433088Z level=info msg="Executing migration" id="create migration_log table" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.94499993Z level=info msg="Executing migration" id="create user table" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.94546006Z level=info msg="Executing migration" id="add unique index user.login" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.945966099Z level=info msg="Executing migration" id="add unique index user.email" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.946631776Z level=info msg="Executing migration" id="drop index UQE_user_login - v1" 2024-09-10T18:55:02.033 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.947088206Z level=info msg="Executing migration" id="drop index UQE_user_email - v1" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.947451465Z level=info msg="Executing migration" id="Rename table user to user_v1 - v1" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.948164087Z level=info msg="Executing migration" id="create user table v2" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.948547513Z level=info msg="Executing migration" id="create index UQE_user_login - v2" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.948952214Z level=info msg="Executing migration" id="create index UQE_user_email - v2" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.949372438Z level=info msg="Executing migration" id="copy data_source v1 to v2" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.949739117Z level=info msg="Executing migration" id="Drop old table user_v1" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.950096209Z level=info msg="Executing migration" id="Add column help_flags1 to user table" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.950487637Z level=info msg="Executing migration" id="Update user table charset" 2024-09-10T18:55:02.034 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.950794101Z level=info msg="Executing migration" id="Add last_seen_at column to user" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.951175075Z level=info msg="Executing migration" id="Add missing user data" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.951511407Z level=info msg="Executing migration" id="Add is_disabled column to user" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.951959743Z level=info msg="Executing migration" id="Add index user.login/user.email" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.952346348Z level=info msg="Executing migration" id="Add is_service_account column to user" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.952805042Z level=info msg="Executing migration" id="Update is_service_account column to nullable" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.953968227Z level=info msg="Executing migration" id="create temp user table v1-7" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.954522937Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v1-7" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.95499263Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v1-7" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.955392705Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v1-7" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.955804126Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v1-7" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.958489103Z level=info msg="Executing migration" id="Update temp_user table charset" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.961065333Z level=info msg="Executing migration" id="drop index IDX_temp_user_email - v1" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.961469076Z level=info msg="Executing migration" id="drop index IDX_temp_user_org_id - v1" 2024-09-10T18:55:02.035 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.964359793Z level=info msg="Executing migration" id="drop index IDX_temp_user_code - v1" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.96711876Z level=info msg="Executing migration" id="drop index IDX_temp_user_status - v1" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.969884365Z level=info msg="Executing migration" id="Rename table temp_user to temp_user_tmp_qwerty - v1" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.9726343Z level=info msg="Executing migration" id="create temp_user v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.975431181Z level=info msg="Executing migration" id="create index IDX_temp_user_email - v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.978041624Z level=info msg="Executing migration" id="create index IDX_temp_user_org_id - v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.978456236Z level=info msg="Executing migration" id="create index IDX_temp_user_code - v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.981088579Z level=info msg="Executing migration" id="create index IDX_temp_user_status - v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.983874049Z level=info msg="Executing migration" id="copy temp_user v1 to v2" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.986637437Z level=info msg="Executing migration" id="drop temp_user_tmp_qwerty" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.989207082Z level=info msg="Executing migration" id="Set created for temp users that will otherwise prematurely expire" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.991833885Z level=info msg="Executing migration" id="create star table" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.994322103Z level=info msg="Executing migration" id="add unique index star.user_id_dashboard_id" 2024-09-10T18:55:02.036 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.99709555Z level=info msg="Executing migration" id="create org table v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:01 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:01.999600969Z level=info msg="Executing migration" id="create index UQE_org_name - v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.002222989Z level=info msg="Executing migration" id="create org_user table v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.004893556Z level=info msg="Executing migration" id="create index IDX_org_user_org_id - v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.007380219Z level=info msg="Executing migration" id="create index UQE_org_user_org_id_user_id - v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.010070167Z level=info msg="Executing migration" id="create index IDX_org_user_user_id - v1" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.010802218Z level=info msg="Executing migration" id="Update org table charset" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.01164166Z level=info msg="Executing migration" id="Update org_user table charset" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.012600181Z level=info msg="Executing migration" id="Migrate all Read Only Viewers to Viewers" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.013426149Z level=info msg="Executing migration" id="create dashboard table" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.014127302Z level=info msg="Executing migration" id="add index dashboard.account_id" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.01493809Z level=info msg="Executing migration" id="add unique index dashboard_account_id_slug" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.015826791Z level=info msg="Executing migration" id="create dashboard_tag table" 2024-09-10T18:55:02.037 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.016532419Z level=info msg="Executing migration" id="add unique index dashboard_tag.dasboard_id_term" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.017465186Z level=info msg="Executing migration" id="drop index UQE_dashboard_tag_dashboard_id_term - v1" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.018145639Z level=info msg="Executing migration" id="Rename table dashboard to dashboard_v1 - v1" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.019021794Z level=info msg="Executing migration" id="create dashboard v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.019886494Z level=info msg="Executing migration" id="create index IDX_dashboard_org_id - v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.020714584Z level=info msg="Executing migration" id="create index UQE_dashboard_org_id_slug - v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.021472553Z level=info msg="Executing migration" id="copy dashboard v1 to v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.022439355Z level=info msg="Executing migration" id="drop table dashboard_v1" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.023109999Z level=info msg="Executing migration" id="alter dashboard.data to mediumtext v1" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.023942253Z level=info msg="Executing migration" id="Add column updated_by in dashboard - v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.024594262Z level=info msg="Executing migration" id="Add column created_by in dashboard - v2" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.025111852Z level=info msg="Executing migration" id="Add column gnetId in dashboard" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.025914335Z level=info msg="Executing migration" id="Add index for gnetId in dashboard" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.026337629Z level=info msg="Executing migration" id="Add column plugin_id in dashboard" 2024-09-10T18:55:02.038 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.026825773Z level=info msg="Executing migration" id="Add index for plugin_id in dashboard" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.027613188Z level=info msg="Executing migration" id="Add index for dashboard_id in dashboard_tag" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.028580021Z level=info msg="Executing migration" id="Update dashboard table charset" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.029337071Z level=info msg="Executing migration" id="Update dashboard_tag table charset" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.03021878Z level=info msg="Executing migration" id="Add column folder_id in dashboard" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.030992793Z level=info msg="Executing migration" id="Add column isFolder in dashboard" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.031707595Z level=info msg="Executing migration" id="Add column has_acl in dashboard" 2024-09-10T18:55:02.039 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.032530141Z level=info msg="Executing migration" id="Add column uid in dashboard" 2024-09-10T18:55:02.281 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:02.281 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:02.281 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:55:02.281 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:55:02.281 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-10T18:55:02.281 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-10T18:55:02.282 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:55:02.282 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.282 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.282 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.282 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.033365905Z level=info msg="Executing migration" id="Update uid column values in dashboard" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.03426037Z level=info msg="Executing migration" id="Add unique index dashboard_org_id_uid" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.035113611Z level=info msg="Executing migration" id="Remove unique index org_id_slug" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.035967035Z level=info msg="Executing migration" id="Update dashboard title length" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.036819352Z level=info msg="Executing migration" id="Add unique index for dashboard_org_id_title_folder_id" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.037622497Z level=info msg="Executing migration" id="create dashboard_provisioning" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.040369061Z level=info msg="Executing migration" id="Rename table dashboard_provisioning to dashboard_provisioning_tmp_qwerty - v1" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.043305047Z level=info msg="Executing migration" id="create dashboard_provisioning v2" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.046100041Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id - v2" 2024-09-10T18:55:02.283 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.048844352Z level=info msg="Executing migration" id="create index IDX_dashboard_provisioning_dashboard_id_name - v2" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.051465476Z level=info msg="Executing migration" id="copy dashboard_provisioning v1 to v2" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.054060144Z level=info msg="Executing migration" id="drop dashboard_provisioning_tmp_qwerty" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.057098627Z level=info msg="Executing migration" id="Add check_sum column" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.059829213Z level=info msg="Executing migration" id="Add index for dashboard_title" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.06262434Z level=info msg="Executing migration" id="delete tags for deleted dashboards" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.065370877Z level=info msg="Executing migration" id="delete stars for deleted dashboards" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.068070633Z level=info msg="Executing migration" id="Add index for dashboard_is_folder" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.070942747Z level=info msg="Executing migration" id="Add isPublic for dashboard" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.073745777Z level=info msg="Executing migration" id="create data_source table" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.07630586Z level=info msg="Executing migration" id="add index data_source.account_id" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.07912807Z level=info msg="Executing migration" id="add unique index data_source.account_id_name" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.081905379Z level=info msg="Executing migration" id="drop index IDX_data_source_account_id - v1" 2024-09-10T18:55:02.284 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.084534425Z level=info msg="Executing migration" id="drop index UQE_data_source_account_id_name - v1" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.087545735Z level=info msg="Executing migration" id="Rename table data_source to data_source_v1 - v1" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.090367939Z level=info msg="Executing migration" id="create data_source table v2" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.092770056Z level=info msg="Executing migration" id="create index IDX_data_source_org_id - v2" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.095719396Z level=info msg="Executing migration" id="create index UQE_data_source_org_id_name - v2" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.09829859Z level=info msg="Executing migration" id="Drop old table data_source_v1 #2" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.100889768Z level=info msg="Executing migration" id="Add column with_credentials" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.103697427Z level=info msg="Executing migration" id="Add secure json data column" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.106329959Z level=info msg="Executing migration" id="Update data_source table charset" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.109090617Z level=info msg="Executing migration" id="Update initial version to 1" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.111830975Z level=info msg="Executing migration" id="Add read_only data column" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.11436499Z level=info msg="Executing migration" id="Migrate logging ds to loki ds" 2024-09-10T18:55:02.285 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.117041539Z level=info msg="Executing migration" id="Update json_data with nulls" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.11962061Z level=info msg="Executing migration" id="Add uid column" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.122436997Z level=info msg="Executing migration" id="Update uid value" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.124995403Z level=info msg="Executing migration" id="Add unique index datasource_org_id_uid" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.12769518Z level=info msg="Executing migration" id="add unique index datasource_org_id_is_default" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.130555674Z level=info msg="Executing migration" id="create api_key table" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.131025227Z level=info msg="Executing migration" id="add index api_key.account_id" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.131831532Z level=info msg="Executing migration" id="add index api_key.key" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.132654571Z level=info msg="Executing migration" id="add index api_key.account_id_name" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.133567277Z level=info msg="Executing migration" id="drop index IDX_api_key_account_id - v1" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.13438027Z level=info msg="Executing migration" id="drop index UQE_api_key_key - v1" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.13520771Z level=info msg="Executing migration" id="drop index UQE_api_key_account_id_name - v1" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.135806403Z level=info msg="Executing migration" id="Rename table api_key to api_key_v1 - v1" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.136887638Z level=info msg="Executing migration" id="create api_key table v2" 2024-09-10T18:55:02.286 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.137663657Z level=info msg="Executing migration" id="create index IDX_api_key_org_id - v2" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.138469774Z level=info msg="Executing migration" id="create index UQE_api_key_key - v2" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.138894637Z level=info msg="Executing migration" id="create index UQE_api_key_org_id_name - v2" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.13968247Z level=info msg="Executing migration" id="copy api_key v1 to v2" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.140577653Z level=info msg="Executing migration" id="Drop old table api_key_v1" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.141327392Z level=info msg="Executing migration" id="Update api_key table charset" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.141631857Z level=info msg="Executing migration" id="Add expires to api_key table" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.142627878Z level=info msg="Executing migration" id="Add service account foreign key" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.143329398Z level=info msg="Executing migration" id="set service account foreign key to nil if 0" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.144148577Z level=info msg="Executing migration" id="Add last_used_at to api_key table" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.144719655Z level=info msg="Executing migration" id="Add is_revoked column to api_key table" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.14560114Z level=info msg="Executing migration" id="create dashboard_snapshot table v4" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.146530518Z level=info msg="Executing migration" id="drop table dashboard_snapshot_v4 #1" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.14725656Z level=info msg="Executing migration" id="create dashboard_snapshot table v5 #2" 2024-09-10T18:55:02.287 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.147664019Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_key - v5" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.148108631Z level=info msg="Executing migration" id="create index UQE_dashboard_snapshot_delete_key - v5" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.148528922Z level=info msg="Executing migration" id="create index IDX_dashboard_snapshot_user_id - v5" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.148945662Z level=info msg="Executing migration" id="alter dashboard_snapshot to mediumtext v2" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.149225442Z level=info msg="Executing migration" id="Update dashboard_snapshot table charset" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.149519647Z level=info msg="Executing migration" id="Add column external_delete_url to dashboard_snapshots table" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.150132259Z level=info msg="Executing migration" id="Add encrypted dashboard json column" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.150762582Z level=info msg="Executing migration" id="Change dashboard_encrypted column to MEDIUMBLOB" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.151111903Z level=info msg="Executing migration" id="create quota table v1" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.151543497Z level=info msg="Executing migration" id="create index UQE_quota_org_id_user_id_target - v1" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.151997486Z level=info msg="Executing migration" id="Update quota table charset" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.152274184Z level=info msg="Executing migration" id="create plugin_setting table" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.152687922Z level=info msg="Executing migration" id="create index UQE_plugin_setting_org_id_plugin_id - v1" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.153123862Z level=info msg="Executing migration" id="Add column plugin_version to plugin_settings" 2024-09-10T18:55:02.288 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.153702991Z level=info msg="Executing migration" id="Update plugin_setting table charset" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.154014789Z level=info msg="Executing migration" id="create session table" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.15444571Z level=info msg="Executing migration" id="Drop old table playlist table" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.154760996Z level=info msg="Executing migration" id="Drop old table playlist_item table" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.155080584Z level=info msg="Executing migration" id="create playlist table v2" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.155482632Z level=info msg="Executing migration" id="create playlist item table v2" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.155932383Z level=info msg="Executing migration" id="Update playlist table charset" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.15620238Z level=info msg="Executing migration" id="Update playlist_item table charset" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.156475091Z level=info msg="Executing migration" id="drop preferences table v2" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.156842005Z level=info msg="Executing migration" id="drop preferences table v3" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.157129911Z level=info msg="Executing migration" id="create preferences table v3" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.157533478Z level=info msg="Executing migration" id="Update preferences table charset" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.157833269Z level=info msg="Executing migration" id="Add column team_id in preferences" 2024-09-10T18:55:02.289 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.158404354Z level=info msg="Executing migration" id="Update team_id column values in preferences" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.15873518Z level=info msg="Executing migration" id="Add column week_start in preferences" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.159320889Z level=info msg="Executing migration" id="Add column preferences.json_data" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.159946993Z level=info msg="Executing migration" id="alter preferences.json_data to mediumtext v1" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.160259601Z level=info msg="Executing migration" id="Add preferences index org_id" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.160697154Z level=info msg="Executing migration" id="Add preferences index user_id" 2024-09-10T18:55:02.290 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.161155623Z level=info msg="Executing migration" id="create alert table v1" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.162029067Z level=info msg="Executing migration" id="add index alert org_id & id " 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.162515323Z level=info msg="Executing migration" id="add index alert state" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.165252781Z level=info msg="Executing migration" id="add index alert dashboard_id" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.165748163Z level=info msg="Executing migration" id="Create alert_rule_tag table v1" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.166147964Z level=info msg="Executing migration" id="Add unique index alert_rule_tag.alert_id_tag_id" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.166656196Z level=info msg="Executing migration" id="drop index UQE_alert_rule_tag_alert_id_tag_id - v1" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.167113039Z level=info msg="Executing migration" id="Rename table alert_rule_tag to alert_rule_tag_v1 - v1" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.16865201Z level=info msg="Executing migration" id="Create alert_rule_tag table v2" 2024-09-10T18:55:02.291 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.16904759Z level=info msg="Executing migration" id="create index UQE_alert_rule_tag_alert_id_tag_id - Add unique index alert_rule_tag.alert_id_tag_id V2" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.169474924Z level=info msg="Executing migration" id="copy alert_rule_tag v1 to v2" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.169819885Z level=info msg="Executing migration" id="drop table alert_rule_tag_v1" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.17016673Z level=info msg="Executing migration" id="create alert_notification table v1" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.170539071Z level=info msg="Executing migration" id="Add column is_default" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.171175388Z level=info msg="Executing migration" id="Add column frequency" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.171819226Z level=info msg="Executing migration" id="Add column send_reminder" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.172480187Z level=info msg="Executing migration" id="Add column disable_resolve_message" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.173160934Z level=info msg="Executing migration" id="add index alert_notification org_id & name" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.173585499Z level=info msg="Executing migration" id="Update alert table charset" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.173889Z level=info msg="Executing migration" id="Update alert_notification table charset" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.174178776Z level=info msg="Executing migration" id="create notification_journal table v1" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.174566404Z level=info msg="Executing migration" id="add index notification_journal org_id & alert_id & notifier_id" 2024-09-10T18:55:02.292 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.175020148Z level=info msg="Executing migration" id="drop alert_notification_journal" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.175388052Z level=info msg="Executing migration" id="create alert_notification_state table v1" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.175807409Z level=info msg="Executing migration" id="add index alert_notification_state org_id & alert_id & notifier_id" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.178257328Z level=info msg="Executing migration" id="Add for to alert table" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.181082264Z level=info msg="Executing migration" id="Add column uid in alert_notification" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.183867542Z level=info msg="Executing migration" id="Update uid column values in alert_notification" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.186594896Z level=info msg="Executing migration" id="Add unique index alert_notification_org_id_uid" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.189578582Z level=info msg="Executing migration" id="Remove unique index org_id_name" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.190045573Z level=info msg="Executing migration" id="Add column secure_settings in alert_notification" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.192842089Z level=info msg="Executing migration" id="alter alert.settings to mediumtext" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.195492341Z level=info msg="Executing migration" id="Add non-unique index alert_notification_state_alert_id" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.198379992Z level=info msg="Executing migration" id="Add non-unique index alert_rule_tag_alert_id" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.19888609Z level=info msg="Executing migration" id="Drop old annotation table v4" 2024-09-10T18:55:02.293 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.201630538Z level=info msg="Executing migration" id="create annotation table v5" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.20448298Z level=info msg="Executing migration" id="add index annotation 0 v3" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.207203405Z level=info msg="Executing migration" id="add index annotation 1 v3" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.207685846Z level=info msg="Executing migration" id="add index annotation 2 v3" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.210455065Z level=info msg="Executing migration" id="add index annotation 3 v3" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.213343781Z level=info msg="Executing migration" id="add index annotation 4 v3" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.216139068Z level=info msg="Executing migration" id="Update annotation table charset" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.216562008Z level=info msg="Executing migration" id="Add column region_id to annotation table" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.219255523Z level=info msg="Executing migration" id="Drop category_id index" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.222081842Z level=info msg="Executing migration" id="Add column tags to annotation table" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.224643286Z level=info msg="Executing migration" id="Create annotation_tag table v2" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.225193565Z level=info msg="Executing migration" id="Add unique index annotation_tag.annotation_id_tag_id" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.227936137Z level=info msg="Executing migration" id="drop index UQE_annotation_tag_annotation_id_tag_id - v2" 2024-09-10T18:55:02.294 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.230454239Z level=info msg="Executing migration" id="Rename table annotation_tag to annotation_tag_v2 - v2" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.232979238Z level=info msg="Executing migration" id="Create annotation_tag table v3" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.233427264Z level=info msg="Executing migration" id="create index UQE_annotation_tag_annotation_id_tag_id - Add unique index annotation_tag.annotation_id_tag_id V3" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.236318907Z level=info msg="Executing migration" id="copy annotation_tag v2 to v3" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.238725174Z level=info msg="Executing migration" id="drop table annotation_tag_v2" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.241383824Z level=info msg="Executing migration" id="Update alert annotations and set TEXT to empty" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.24174746Z level=info msg="Executing migration" id="Add created time to annotation table" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.244521131Z level=info msg="Executing migration" id="Add updated time to annotation table" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.247057984Z level=info msg="Executing migration" id="Add index for created in annotation table" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.24965787Z level=info msg="Executing migration" id="Add index for updated in annotation table" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.250127445Z level=info msg="Executing migration" id="Convert existing annotations from seconds to milliseconds" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.250853303Z level=info msg="Executing migration" id="Add epoch_end column" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.25166768Z level=info msg="Executing migration" id="Add index for epoch_end" 2024-09-10T18:55:02.295 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.25219827Z level=info msg="Executing migration" id="Make epoch_end the same as epoch" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.252917876Z level=info msg="Executing migration" id="Move region to single row" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.253337755Z level=info msg="Executing migration" id="Remove index org_id_epoch from annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.253770441Z level=info msg="Executing migration" id="Remove index org_id_dashboard_id_panel_id_epoch from annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.254592305Z level=info msg="Executing migration" id="Add index for org_id_dashboard_id_epoch_end_epoch on annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.255566234Z level=info msg="Executing migration" id="Add index for org_id_epoch_end_epoch on annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.256203882Z level=info msg="Executing migration" id="Remove index org_id_epoch_epoch_end from annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.256593702Z level=info msg="Executing migration" id="Add index for alert_id on annotation table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.257051358Z level=info msg="Executing migration" id="Increase tags column to length 4096" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.257441325Z level=info msg="Executing migration" id="create test_data table" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.257889523Z level=info msg="Executing migration" id="create dashboard_version table v1" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.258324594Z level=info msg="Executing migration" id="add index dashboard_version.dashboard_id" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.259121758Z level=info msg="Executing migration" id="add unique index dashboard_version.dashboard_id and dashboard_version.version" 2024-09-10T18:55:02.296 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.25959032Z level=info msg="Executing migration" id="Set dashboard version to 1 where 0" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.260376492Z level=info msg="Executing migration" id="save existing dashboard data in dashboard_version table v1" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.261321227Z level=info msg="Executing migration" id="alter dashboard_version.data to mediumtext v1" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.262069843Z level=info msg="Executing migration" id="create team table" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.262548753Z level=info msg="Executing migration" id="add index team.org_id" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.263550385Z level=info msg="Executing migration" id="add unique index team_org_id_name" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.264222725Z level=info msg="Executing migration" id="create team member table" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.265033085Z level=info msg="Executing migration" id="add index team_member.org_id" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.265485866Z level=info msg="Executing migration" id="add unique index team_member_org_id_team_id_user_id" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.266355265Z level=info msg="Executing migration" id="add index team_member.team_id" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.267328453Z level=info msg="Executing migration" id="Add column email to team table" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.268260874Z level=info msg="Executing migration" id="Add column external to team_member table" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.269047516Z level=info msg="Executing migration" id="Add column permission to team_member table" 2024-09-10T18:55:02.297 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.269902433Z level=info msg="Executing migration" id="create dashboard acl table" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.270729598Z level=info msg="Executing migration" id="add index dashboard_acl_dashboard_id" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.271621651Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_user_id" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.272088585Z level=info msg="Executing migration" id="add unique index dashboard_acl_dashboard_id_team_id" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.272552863Z level=info msg="Executing migration" id="add index dashboard_acl_user_id" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.273387131Z level=info msg="Executing migration" id="add index dashboard_acl_team_id" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.273859424Z level=info msg="Executing migration" id="add index dashboard_acl_org_id_role" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.274314869Z level=info msg="Executing migration" id="add index dashboard_permission" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.274795826Z level=info msg="Executing migration" id="save default acl rules in dashboard_acl table" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.275231976Z level=info msg="Executing migration" id="delete acl rules for deleted dashboards and folders" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.275536541Z level=info msg="Executing migration" id="create tag table" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.275957373Z level=info msg="Executing migration" id="add index tag.key_value" 2024-09-10T18:55:02.298 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.276411006Z level=info msg="Executing migration" id="create login attempt table" 2024-09-10T18:55:02.299 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.276848727Z level=info msg="Executing migration" id="add index login_attempt.username" 2024-09-10T18:55:02.299 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.277300001Z level=info msg="Executing migration" id="drop index IDX_login_attempt_username - v1" 2024-09-10T18:55:02.299 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.277733358Z level=info msg="Executing migration" id="Rename table login_attempt to login_attempt_tmp_qwerty - v1" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.279639218Z level=info msg="Executing migration" id="create login_attempt v2" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.280081167Z level=info msg="Executing migration" id="create index IDX_login_attempt_username - v2" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.280548265Z level=info msg="Executing migration" id="copy login_attempt v1 to v2" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.280940873Z level=info msg="Executing migration" id="drop login_attempt_tmp_qwerty" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.281310134Z level=info msg="Executing migration" id="create user auth table" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.281719889Z level=info msg="Executing migration" id="create index IDX_user_auth_auth_module_auth_id - v1" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.282218488Z level=info msg="Executing migration" id="alter user_auth.auth_id to length 190" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.282536151Z level=info msg="Executing migration" id="Add OAuth access token to user_auth" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.283419556Z level=info msg="Executing migration" id="Add OAuth refresh token to user_auth" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.284400937Z level=info msg="Executing migration" id="Add OAuth token type to user_auth" 2024-09-10T18:55:02.300 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.285403913Z level=info msg="Executing migration" id="Add OAuth expiry to user_auth" 2024-09-10T18:55:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:55:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: pgmap v33: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [7, 2]}]': finished 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [1, 2]}]': finished 2024-09-10T18:55:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [1, 5]}]': finished 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.17", "id": [1, 2]}]': finished 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: osdmap e62: 8 total, 8 up, 8 in 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.28631734Z level=info msg="Executing migration" id="Add index to user_id column in user_auth" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.287138839Z level=info msg="Executing migration" id="Add OAuth ID token to user_auth" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.287992196Z level=info msg="Executing migration" id="create server_lock table" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.288863849Z level=info msg="Executing migration" id="add index server_lock.operation_uid" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.289689343Z level=info msg="Executing migration" id="create user auth token table" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.290471515Z level=info msg="Executing migration" id="add unique index user_auth_token.auth_token" 2024-09-10T18:55:02.533 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.291353246Z level=info msg="Executing migration" id="add unique index user_auth_token.prev_auth_token" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.294157388Z level=info msg="Executing migration" id="add index user_auth_token.user_id" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.296996505Z level=info msg="Executing migration" id="Add revoked_at to the user auth token" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.299772224Z level=info msg="Executing migration" id="create cache_data table" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.302568187Z level=info msg="Executing migration" id="add unique index cache_data.cache_key" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.305155556Z level=info msg="Executing migration" id="create short_url table v1" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.307856624Z level=info msg="Executing migration" id="add index short_url.org_id-uid" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.310798821Z level=info msg="Executing migration" id="alter table short_url alter column created_by type to bigint" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.313476461Z level=info msg="Executing migration" id="delete alert_definition table" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.316229254Z level=info msg="Executing migration" id="recreate alert_definition table" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.31911792Z level=info msg="Executing migration" id="add index in alert_definition on org_id and title columns" 2024-09-10T18:55:02.534 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.321696894Z level=info msg="Executing migration" id="add index in alert_definition on org_id and uid columns" 2024-09-10T18:55:02.535 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.322194303Z level=info msg="Executing migration" id="alter alert_definition table data column to mediumtext in mysql" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.325016128Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and title columns" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.325501375Z level=info msg="Executing migration" id="drop index in alert_definition on org_id and uid columns" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.327823441Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and title columns" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.330751259Z level=info msg="Executing migration" id="add unique index in alert_definition on org_id and uid columns" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.331238572Z level=info msg="Executing migration" id="Add column paused in alert_definition" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.333913531Z level=info msg="Executing migration" id="drop alert_definition table" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.336629707Z level=info msg="Executing migration" id="delete alert_definition_version table" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.339567991Z level=info msg="Executing migration" id="recreate alert_definition_version table" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.340085242Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_id and version columns" 2024-09-10T18:55:02.536 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.342816803Z level=info msg="Executing migration" id="add index in alert_definition_version table on alert_definition_uid and version columns" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.343440449Z level=info msg="Executing migration" id="alter alert_definition_version table data column to mediumtext in mysql" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.345493483Z level=info msg="Executing migration" id="drop alert_definition_version table" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.348108681Z level=info msg="Executing migration" id="create alert_instance table" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.34860676Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, def_uid and current_state columns" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.351402839Z level=info msg="Executing migration" id="add index in alert_instance table on def_org_id, current_state columns" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.354004674Z level=info msg="Executing migration" id="add column current_state_end to alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.356668588Z level=info msg="Executing migration" id="remove index def_org_id, def_uid, current_state on alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.357102512Z level=info msg="Executing migration" id="remove index def_org_id, current_state on alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.359720407Z level=info msg="Executing migration" id="rename def_org_id to rule_org_id in alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.363292855Z level=info msg="Executing migration" id="rename def_uid to rule_uid in alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.366809713Z level=info msg="Executing migration" id="add index rule_org_id, rule_uid, current_state on alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.36930465Z level=info msg="Executing migration" id="add index rule_org_id, current_state on alert_instance" 2024-09-10T18:55:02.537 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.369792089Z level=info msg="Executing migration" id="add current_reason column related to current_state" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.370613369Z level=info msg="Executing migration" id="create alert_rule table" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.37141492Z level=info msg="Executing migration" id="add index in alert_rule on org_id and title columns" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.372283329Z level=info msg="Executing migration" id="add index in alert_rule on org_id and uid columns" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.37277612Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespace_uid, group_uid columns" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.373602677Z level=info msg="Executing migration" id="alter alert_rule table data column to mediumtext in mysql" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.374554374Z level=info msg="Executing migration" id="add column for to alert_rule" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.375581758Z level=info msg="Executing migration" id="add column annotations to alert_rule" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.376406369Z level=info msg="Executing migration" id="add column labels to alert_rule" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.377270934Z level=info msg="Executing migration" id="remove unique index from alert_rule on org_id, title columns" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.37806983Z level=info msg="Executing migration" id="add index in alert_rule on org_id, namespase_uid and title columns" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.378898635Z level=info msg="Executing migration" id="add dashboard_uid column to alert_rule" 2024-09-10T18:55:02.538 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.379765707Z level=info msg="Executing migration" id="add panel_id column to alert_rule" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.380626135Z level=info msg="Executing migration" id="add index in alert_rule on org_id, dashboard_uid and panel_id columns" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.381400315Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.382248853Z level=info msg="Executing migration" id="add is_paused column to alert_rule table" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.38311816Z level=info msg="Executing migration" id="fix is_paused column for alert_rule table" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.383924904Z level=info msg="Executing migration" id="create alert_rule_version table" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.38478411Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_uid and version columns" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.385644662Z level=info msg="Executing migration" id="add index in alert_rule_version table on rule_org_id, rule_namespace_uid and rule_group columns" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.386542706Z level=info msg="Executing migration" id="alter alert_rule_version table data column to mediumtext in mysql" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.387259778Z level=info msg="Executing migration" id="add column for to alert_rule_version" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.388184177Z level=info msg="Executing migration" id="add column annotations to alert_rule_version" 2024-09-10T18:55:02.539 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.389072831Z level=info msg="Executing migration" id="add column labels to alert_rule_version" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.389977084Z level=info msg="Executing migration" id="add rule_group_idx column to alert_rule_version" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.390876079Z level=info msg="Executing migration" id="add is_paused column to alert_rule_versions table" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.391756781Z level=info msg="Executing migration" id="fix is_paused column for alert_rule_version table" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.392597706Z level=info msg="Executing migration" id=create_alert_configuration_table 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.393013615Z level=info msg="Executing migration" id="Add column default in alert_configuration" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.3938918Z level=info msg="Executing migration" id="alert alert_configuration alertmanager_configuration column from TEXT to MEDIUMTEXT if mysql" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.394778429Z level=info msg="Executing migration" id="add column org_id in alert_configuration" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.395854461Z level=info msg="Executing migration" id="add index in alert_configuration table on org_id column" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.39631253Z level=info msg="Executing migration" id="add configuration_hash column to alert_configuration" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.397198105Z level=info msg="Executing migration" id=create_ngalert_configuration_table 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.398096289Z level=info msg="Executing migration" id="add index in ngalert_configuration on org_id column" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.399089659Z level=info msg="Executing migration" id="add column send_alerts_to in ngalert_configuration" 2024-09-10T18:55:02.540 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.400056102Z level=info msg="Executing migration" id="create provenance_type table" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.402977576Z level=info msg="Executing migration" id="add index to uniquify (record_key, record_type, org_id) columns" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.405658702Z level=info msg="Executing migration" id="create alert_image table" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.40855783Z level=info msg="Executing migration" id="add unique index on token to alert_image table" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.409039847Z level=info msg="Executing migration" id="support longer URLs in alert_image table" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.411626117Z level=info msg="Executing migration" id=create_alert_configuration_history_table 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.412080576Z level=info msg="Executing migration" id="drop non-unique orgID index on alert_configuration" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.414276145Z level=info msg="Executing migration" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.41437819Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop unique orgID index on alert_configuration if exists" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.414683473Z level=info msg="Executing migration" id="extract alertmanager configuration history to separate table" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.417168674Z level=info msg="Executing migration" id="add unique index on orgID to alert_configuration" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.417675674Z level=info msg="Executing migration" id="clear migration entry \"remove unified alerting data\"" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.41781199Z level=info msg="Executing migration" id="move dashboard alerts to unified alerting" 2024-09-10T18:55:02.541 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.417932562Z level=info msg="alerts found to migrate" alerts=0 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.420478813Z level=info msg="Executing migration" id="create library_element table v1" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.423155761Z level=info msg="Executing migration" id="add index library_element org_id-folder_id-name-kind" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.426008634Z level=info msg="Executing migration" id="create library_element_connection table v1" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.426529546Z level=info msg="Executing migration" id="add index library_element_connection element_id-kind-connection_id" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.429172522Z level=info msg="Executing migration" id="add unique index library_element org_id_uid" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.431830658Z level=info msg="Executing migration" id="increase max description length to 2048" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.434852294Z level=info msg="Executing migration" id="clone move dashboard alerts to unified alerting" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.435271851Z level=info msg="Executing migration" id="create data_keys table" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.437926887Z level=info msg="Executing migration" id="create secrets table" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.440536134Z level=info msg="Executing migration" id="rename data_keys name column to id" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.445087145Z level=info msg="Executing migration" id="add name column into data_keys" 2024-09-10T18:55:02.542 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.446064551Z level=info msg="Executing migration" id="copy data_keys id column values into name" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.448685621Z level=info msg="Executing migration" id="rename data_keys name column to label" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.452924568Z level=info msg="Executing migration" id="rename data_keys id column back to name" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.457259182Z level=info msg="Executing migration" id="create kv_store table v1" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.457915552Z level=info msg="Executing migration" id="add index kv_store.org_id-namespace-key" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.460648494Z level=info msg="Executing migration" id="update dashboard_uid and panel_id from existing annotations" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.463132161Z level=info msg="Executing migration" id="create permission table" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.465727673Z level=info msg="Executing migration" id="add unique index permission.role_id" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.466203645Z level=info msg="Executing migration" id="add unique index role_id_action_scope" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.46891611Z level=info msg="Executing migration" id="create role table" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.47146193Z level=info msg="Executing migration" id="add column display_name" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.473828567Z level=info msg="Executing migration" id="add column group_name" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.474767369Z level=info msg="Executing migration" id="add index role.org_id" 2024-09-10T18:55:02.543 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.47568005Z level=info msg="Executing migration" id="add unique index role_org_id_name" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.476528452Z level=info msg="Executing migration" id="add index role_org_id_uid" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.477251787Z level=info msg="Executing migration" id="create team role table" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.477682872Z level=info msg="Executing migration" id="add index team_role.org_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.478478009Z level=info msg="Executing migration" id="add unique index team_role_org_id_team_id_role_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.479365288Z level=info msg="Executing migration" id="add index team_role.team_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.480092798Z level=info msg="Executing migration" id="create user role table" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.480535031Z level=info msg="Executing migration" id="add index user_role.org_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.481308304Z level=info msg="Executing migration" id="add unique index user_role_org_id_user_id_role_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.482213167Z level=info msg="Executing migration" id="add index user_role.user_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.482876034Z level=info msg="Executing migration" id="create builtin role table" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.483320584Z level=info msg="Executing migration" id="add index builtin_role.role_id" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.484198568Z level=info msg="Executing migration" id="add index builtin_role.name" 2024-09-10T18:55:02.544 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.485141951Z level=info msg="Executing migration" id="Add column org_id to builtin_role table" 2024-09-10T18:55:02.545 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.486296236Z level=info msg="Executing migration" id="add index builtin_role.org_id" 2024-09-10T18:55:02.545 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.48684185Z level=info msg="Executing migration" id="add unique index builtin_role_org_id_role_id_role" 2024-09-10T18:55:02.545 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.487635023Z level=info msg="Executing migration" id="Remove unique index role_org_id_uid" 2024-09-10T18:55:02.545 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.488608762Z level=info msg="Executing migration" id="add unique index role.uid" 2024-09-10T18:55:02.545 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.489320203Z level=info msg="Executing migration" id="create seed assignment table" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.48974019Z level=info msg="Executing migration" id="add unique index builtin_role_role_name" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.490207507Z level=info msg="Executing migration" id="add column hidden to role table" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.491214247Z level=info msg="Executing migration" id="create query_history table v1" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.491665108Z level=info msg="Executing migration" id="add index query_history.org_id-created_by-datasource_uid" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.49213042Z level=info msg="Executing migration" id="alter table query_history alter column created_by type to bigint" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.492422993Z level=info msg="Executing migration" id="teams permissions migration" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.492891692Z level=info msg="Executing migration" id="dashboard permissions" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.493402066Z level=info msg="Executing migration" id="dashboard permissions uid scopes" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.493829207Z level=info msg="Executing migration" id="drop managed folder create actions" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.494185331Z level=info msg="Executing migration" id="alerting notification permissions" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.494550589Z level=info msg="Executing migration" id="create query_history_star table v1" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.494965992Z level=info msg="Executing migration" id="add index query_history.user_id-query_uid" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.495439163Z level=info msg="Executing migration" id="add column org_id in query_history_star" 2024-09-10T18:55:02.546 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.49653245Z level=info msg="Executing migration" id="alter table query_history_star_mig column user_id type to bigint" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.496852555Z level=info msg="Executing migration" id="create correlation table v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.497283197Z level=info msg="Executing migration" id="add index correlations.uid" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.49774193Z level=info msg="Executing migration" id="add index correlations.source_uid" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.498182175Z level=info msg="Executing migration" id="add correlation config column" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.499196281Z level=info msg="Executing migration" id="create entity_events table" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.499647413Z level=info msg="Executing migration" id="create dashboard public config v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.502465452Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.502584632Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.502898279Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.502987207Z level=warn msg="Skipping migration: Already executed, but not recorded in migration log" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.503252212Z level=info msg="Executing migration" id="Drop old dashboard public config table" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.503719105Z level=info msg="Executing migration" id="recreate dashboard public config v1" 2024-09-10T18:55:02.547 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.504149578Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v1" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.506867489Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v1" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.509574962Z level=info msg="Executing migration" id="drop index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.512153914Z level=info msg="Executing migration" id="drop index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.514790548Z level=info msg="Executing migration" id="Drop public config table" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.517515584Z level=info msg="Executing migration" id="Recreate dashboard public config v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.520059285Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_uid - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.522718801Z level=info msg="Executing migration" id="create index IDX_dashboard_public_config_org_id_dashboard_uid - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.523200366Z level=info msg="Executing migration" id="create index UQE_dashboard_public_config_access_token - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.526045245Z level=info msg="Executing migration" id="Rename table dashboard_public_config to dashboard_public - v2" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.529170114Z level=info msg="Executing migration" id="add annotations_enabled column" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.53191054Z level=info msg="Executing migration" id="add time_selection_enabled column" 2024-09-10T18:55:02.548 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.533450232Z level=info msg="Executing migration" id="delete orphaned public dashboards" 2024-09-10T18:55:02.549 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.536100947Z level=info msg="Executing migration" id="add share column" 2024-09-10T18:55:02.892 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:02 smithi107 systemd[1]: Starting Ceph node-exporter.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.53868321Z level=info msg="Executing migration" id="create default alerting folders" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.541332633Z level=info msg="Executing migration" id="create file table" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.541875243Z level=info msg="Executing migration" id="file table idx: path natural pk" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.542652346Z level=info msg="Executing migration" id="file table idx: parent_folder_path_hash fast folder retrieval" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.543458876Z level=info msg="Executing migration" id="create file_meta table" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.543862793Z level=info msg="Executing migration" id="file table idx: path key" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.544306453Z level=info msg="Executing migration" id="set path collation in file table" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.545145095Z level=info msg="Executing migration" id="managed permissions migration" 2024-09-10T18:55:02.925 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.545770128Z level=info msg="Executing migration" id="managed folder permissions alert actions migration" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.546109017Z level=info msg="Executing migration" id="RBAC action name migrator" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.546947607Z level=info msg="Executing migration" id="Add UID column to playlist" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.548289054Z level=info msg="Executing migration" id="Update uid column values in playlist" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.548626781Z level=info msg="Executing migration" id="Add index for uid in playlist" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.549476154Z level=info msg="Executing migration" id="update group index for alert rules" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.550321871Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated migration" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.551228119Z level=info msg="Executing migration" id="admin only folder/dashboard permission" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.551813973Z level=info msg="Executing migration" id="add action column to seed_assignment" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.552855502Z level=info msg="Executing migration" id="add scope column to seed_assignment" 2024-09-10T18:55:02.926 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.553896859Z level=info msg="Executing migration" id="remove unique index builtin_role_role_name before nullable update" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.55464787Z level=info msg="Executing migration" id="update seed_assignment role_name column to nullable" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.56429438Z level=info msg="Executing migration" id="add unique index builtin_role_name back" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.565156727Z level=info msg="Executing migration" id="add unique index builtin_role_action_scope" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.566117655Z level=info msg="Executing migration" id="add primary key to seed_assigment" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.569532906Z level=info msg="Executing migration" id="managed folder permissions alert actions repeated fixed migration" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.569868998Z level=info msg="Executing migration" id="migrate external alertmanagers to datsourcse" 2024-09-10T18:55:02.927 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.570682035Z level=info msg="Executing migration" id="create folder table" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.571113211Z level=info msg="Executing migration" id="Add index for parent_uid" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.571564283Z level=info msg="Executing migration" id="Add unique index for folder.uid and folder.org_id" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.572540629Z level=info msg="Executing migration" id="Update folder title length" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.572918806Z level=info msg="Executing migration" id="Add unique index for folder.title and folder.parent_uid" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=migrator t=2024-09-10T18:55:02.573806948Z level=info msg="migrations completed" performed=485 skipped=0 duration=629.398065ms 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore t=2024-09-10T18:55:02.5742162Z level=info msg="Created default organization" 2024-09-10T18:55:02.928 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=plugin.loader t=2024-09-10T18:55:02.630117232Z level=info msg="Plugin registered" pluginID=input 2024-09-10T18:55:02.929 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=plugin.loader t=2024-09-10T18:55:02.65730317Z level=info msg="Plugin registered" pluginID=grafana-piechart-panel 2024-09-10T18:55:02.929 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=plugin.loader t=2024-09-10T18:55:02.657340884Z level=info msg="Plugin registered" pluginID=vonage-status-panel 2024-09-10T18:55:02.929 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=secrets t=2024-09-10T18:55:02.657375866Z level=info msg="Envelope encryption state" enabled=true currentprovider=secretKey.v1 2024-09-10T18:55:02.929 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=query_data t=2024-09-10T18:55:02.659109986Z level=info msg="Query Service initialization" 2024-09-10T18:55:02.929 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=live.push_http t=2024-09-10T18:55:02.661814808Z level=info msg="Live Push Gateway initialization" 2024-09-10T18:55:03.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[29404]: Deploying daemon node-exporter.a on smithi107 2024-09-10T18:55:03.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:03.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[29404]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:55:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[40355]: Deploying daemon node-exporter.a on smithi107 2024-09-10T18:55:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:03 smithi107 ceph-mon[40355]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:55:03.216 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:02 smithi107 bash[77772]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:55:03.393 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-mon[33228]: Deploying daemon node-exporter.a on smithi107 2024-09-10T18:55:03.394 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:03.394 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-mon[33228]: osdmap e63: 8 total, 8 up, 8 in 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=infra.usagestats.collector t=2024-09-10T18:55:03.393186111Z level=info msg="registering usage stat providers" usageStatsProvidersLen=2 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=provisioning.datasources t=2024-09-10T18:55:03.393608282Z level=warn msg="[Deprecated] the datasource provisioning config is outdated. please upgrade" filename=/etc/grafana/provisioning/datasources/ceph-dashboard.yml 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=provisioning.datasources t=2024-09-10T18:55:03.426066498Z level=info msg="inserting datasource from configuration " name=Dashboard1 uid=P43CA22E17D0F9596 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=provisioning.datasources t=2024-09-10T18:55:03.458124038Z level=info msg="inserting datasource from configuration " name=Loki uid=P8E80F9AEF21F6940 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=provisioning.alerting t=2024-09-10T18:55:03.479531591Z level=info msg="starting to provision alerting" 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=provisioning.alerting t=2024-09-10T18:55:03.479552588Z level=info msg="finished to provision alerting" 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=grafanaStorageLogger t=2024-09-10T18:55:03.479645285Z level=info msg="storage starting" 2024-09-10T18:55:03.674 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=ngalert.state.manager t=2024-09-10T18:55:03.479716474Z level=info msg="Warming state cache for startup" 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=http.server t=2024-09-10T18:55:03.481569783Z level=info msg="HTTP Server Listen" address=[::]:3000 protocol=https subUrl= socket= 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=ngalert.state.manager t=2024-09-10T18:55:03.490042392Z level=info msg="State cache has been initialized" states=0 duration=10.32117ms 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T18:55:03.490070748Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=ngalert.multiorg.alertmanager t=2024-09-10T18:55:03.490213728Z level=info msg="starting MultiOrg Alertmanager" 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=ticker t=2024-09-10T18:55:03.490238647Z level=info msg=starting first_tick=2024-09-10T18:55:10Z 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T18:55:03.526678871Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T18:55:03.567765975Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T18:55:03.675 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 18:55:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T18:55:03.578310494Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=1 code="database is locked" 2024-09-10T18:55:03.715 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:03 smithi107 bash[77772]: Getting image source signatures 2024-09-10T18:55:03.715 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:03 smithi107 bash[77772]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:55:03.715 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:03 smithi107 bash[77772]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:55:03.716 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:03 smithi107 bash[77772]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:55:04.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:04 smithi162 ceph-mon[33228]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:04 smithi162 ceph-mon[33228]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:55:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:04 smithi107 ceph-mon[29404]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:04 smithi107 ceph-mon[29404]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:55:04.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:04 smithi107 ceph-mon[40355]: pgmap v36: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:04.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:04 smithi107 ceph-mon[40355]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-09-10T18:55:05.185 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:04 smithi107 bash[77772]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:55:05.185 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 bash[77772]: Writing manifest to image destination 2024-09-10T18:55:05.764 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 podman[77772]: 2024-09-10 18:55:05.457057623 +0000 UTC m=+2.670575244 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:55:05.764 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 podman[77772]: 2024-09-10 18:55:05.572344335 +0000 UTC m=+2.785861955 container create 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:06.135 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 podman[77772]: 2024-09-10 18:55:05.849750986 +0000 UTC m=+3.063268617 container init 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 podman[77772]: 2024-09-10 18:55:05.85544138 +0000 UTC m=+3.068959010 container start 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.859Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.859Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.859Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:55:06.136 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.860Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:55:06.137 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:55:06.138 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:55:06.139 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a[77847]: ts=2024-09-10T18:55:05.861Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:55:06.140 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 bash[77772]: 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 2024-09-10T18:55:06.141 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 18:55:05 smithi107 systemd[1]: Started Ceph node-exporter.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:06 smithi162 ceph-mon[33228]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:06 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:06 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:06 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:06 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[40355]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:06.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[29404]: pgmap v37: 132 pgs: 4 peering, 128 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:06 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:06.924 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:06 smithi162 systemd[1]: Starting Ceph node-exporter.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:07.287 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:07 smithi162 ceph-mon[33228]: Deploying daemon node-exporter.b on smithi162 2024-09-10T18:55:07.287 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:06 smithi162 bash[67139]: Trying to pull quay.io/prometheus/node-exporter:v1.5.0... 2024-09-10T18:55:07.287 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:07 smithi162 bash[67139]: Getting image source signatures 2024-09-10T18:55:07.287 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:07 smithi162 bash[67139]: Copying blob sha256:5c12815fee558b157dee7f7509dedbaba0a8379098858a65ec869e1f1526ea0c 2024-09-10T18:55:07.287 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:07 smithi162 bash[67139]: Copying blob sha256:c0e87333d3805a7b7e1c25783dd25f27da0fe70a8e262eee7ed190c8dc843fa7 2024-09-10T18:55:07.287 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:07 smithi162 bash[67139]: Copying blob sha256:22b70bddd3acadc892fca4c2af4260629bfda5dfd11ebc106a93ce24e752b5ed 2024-09-10T18:55:07.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:07 smithi107 ceph-mon[29404]: Deploying daemon node-exporter.b on smithi162 2024-09-10T18:55:07.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:07 smithi107 ceph-mon[40355]: Deploying daemon node-exporter.b on smithi162 2024-09-10T18:55:08.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:08 smithi162 ceph-mon[33228]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:08 smithi162 ceph-mon[33228]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:55:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:08 smithi162 ceph-mon[33228]: Cluster is now healthy 2024-09-10T18:55:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[29404]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[29404]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:55:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[29404]: Cluster is now healthy 2024-09-10T18:55:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[40355]: pgmap v38: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[40355]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-09-10T18:55:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:08 smithi107 ceph-mon[40355]: Cluster is now healthy 2024-09-10T18:55:09.174 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:08 smithi162 bash[67139]: Copying config sha256:0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 2024-09-10T18:55:09.174 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:08 smithi162 bash[67139]: Writing manifest to image destination 2024-09-10T18:55:09.561 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 podman[67139]: 2024-09-10 18:55:09.208948412 +0000 UTC m=+2.370284280 image pull 0da6a335fe1356545476b749c68f022c897de3a2139e8f0054f6937349ee2b83 quay.io/prometheus/node-exporter:v1.5.0 2024-09-10T18:55:09.561 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 podman[67139]: 2024-09-10 18:55:09.282669667 +0000 UTC m=+2.444005576 container create 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:55:09.561 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 podman[67139]: 2024-09-10 18:55:09.476635209 +0000 UTC m=+2.637971097 container init 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:55:09.561 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 podman[67139]: 2024-09-10 18:55:09.481759457 +0000 UTC m=+2.643095327 container start 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T18:55:09.561 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=node_exporter.go:180 level=info msg="Starting node_exporter" version="(version=1.5.0, branch=HEAD, revision=1b48970ffcf5630534fb00bb0687d73c66d1c959)" 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=node_exporter.go:181 level=info msg="Build context" build_context="(go=go1.19.3, user=root@6e7732a7b81b, date=20221129-18:59:09)" 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=diskstats_common.go:111 level=info collector=diskstats msg="Parsed flag --collector.diskstats.device-exclude" flag=^(ram|loop|fd|(h|s|v|xv)d[a-z]|nvme\d+n\d+p)\d+$ 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=diskstats_linux.go:264 level=error collector=diskstats msg="Failed to open directory, disabling udev device properties" path=/run/udev/data 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=filesystem_common.go:111 level=info collector=filesystem msg="Parsed flag --collector.filesystem.mount-points-exclude" flag=^/(dev|proc|run/credentials/.+|sys|var/lib/docker/.+|var/lib/containers/storage/.+)($|/) 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.485Z caller=filesystem_common.go:113 level=info collector=filesystem msg="Parsed flag --collector.filesystem.fs-types-exclude" flag=^(autofs|binfmt_misc|bpf|cgroup2?|configfs|debugfs|devpts|devtmpfs|fusectl|hugetlbfs|iso9660|mqueue|nsfs|overlay|proc|procfs|pstore|rpc_pipefs|securityfs|selinuxfs|squashfs|sysfs|tracefs)$ 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:110 level=info msg="Enabled collectors" 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=arp 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=bcache 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=bonding 2024-09-10T18:55:09.562 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=btrfs 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=conntrack 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=cpu 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=cpufreq 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=diskstats 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=dmi 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=edac 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=entropy 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=fibrechannel 2024-09-10T18:55:09.563 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=filefd 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=filesystem 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=hwmon 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=infiniband 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=ipvs 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=loadavg 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=mdadm 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=meminfo 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=netclass 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=netdev 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=netstat 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=nfs 2024-09-10T18:55:09.564 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=nfsd 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=nvme 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=os 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=powersupplyclass 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=pressure 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=rapl 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=schedstat 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=selinux 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=sockstat 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=softnet 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=stat 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=tapestats 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=textfile 2024-09-10T18:55:09.565 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=thermal_zone 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=time 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=udp_queues 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=uname 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=vmstat 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=xfs 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=node_exporter.go:117 level=info collector=zfs 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9100 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b[67210]: ts=2024-09-10T18:55:09.486Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9100 2024-09-10T18:55:09.566 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 bash[67139]: 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 2024-09-10T18:55:09.923 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 18:55:09 smithi162 systemd[1]: Started Ceph node-exporter.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: pgmap v39: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s; 13 B/s, 1 objects/s recovering 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:10.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:10 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:55:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:12 smithi162 ceph-mon[33228]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:55:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:12 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:12.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:12.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:12 smithi107 ceph-mon[29404]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:55:12.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:12 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:12 smithi107 ceph-mon[40355]: pgmap v40: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s; 11 B/s, 1 objects/s recovering 2024-09-10T18:55:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:12 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:13.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:13.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:13.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:14 smithi162 ceph-mon[33228]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.8 KiB/s rd, 93 B/s wr, 5 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:55:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:14 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:14 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[29404]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.8 KiB/s rd, 93 B/s wr, 5 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:55:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[40355]: pgmap v41: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.8 KiB/s rd, 93 B/s wr, 5 op/s; 10 B/s, 1 objects/s recovering 2024-09-10T18:55:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:14 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:15.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:15 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:15 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:15 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:15 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:15 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:15 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:16.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[29404]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:55:16.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[29404]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-10T18:55:16.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[29404]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[40355]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[40355]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[40355]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:16.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:16 smithi162 ceph-mon[33228]: Reconfiguring alertmanager.a (dependencies changed)... 2024-09-10T18:55:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:16 smithi162 ceph-mon[33228]: Reconfiguring daemon alertmanager.a on smithi107 2024-09-10T18:55:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:16 smithi162 ceph-mon[33228]: pgmap v42: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 2.6 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:18.337 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:18 smithi107 systemd[1]: Stopping Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:18 smithi162 ceph-mon[33228]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:18 smithi107 ceph-mon[29404]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:18.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:18 smithi107 ceph-mon[40355]: pgmap v43: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 85 B/s wr, 5 op/s; 9 B/s, 0 objects/s recovering 2024-09-10T18:55:18.716 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:18 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[77008]: ts=2024-09-10T18:55:18.513Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:55:18.716 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:18 smithi107 podman[78662]: 2024-09-10 18:55:18.534272284 +0000 UTC m=+0.144095594 container died 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:19.215 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78662]: 2024-09-10 18:55:19.001260301 +0000 UTC m=+0.611083613 container cleanup 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:19.215 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 bash[78662]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T18:55:19.563 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78679]: 2024-09-10 18:55:19.260774758 +0000 UTC m=+0.720825636 container remove 3ad003d0a9cb86ae3ff6afd32660473b0b38597e7566e58096b1a684fb5093d8 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:19.563 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78679]: 2024-09-10 18:55:19.336471931 +0000 UTC m=+0.796522817 volume remove 41ecb289618ad3dc0dc1662d18109f78b30fae8c9a1689d896d178805ad9affd 2024-09-10T18:55:19.825 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T18:55:19.825 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 systemd[1]: Stopped Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:19.825 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.170s CPU time. 2024-09-10T18:55:19.825 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 systemd[1]: Starting Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:20.140 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78762]: 2024-09-10 18:55:19.727300353 +0000 UTC m=+0.022747456 image pull c8568f914cd25b2062c44e9f79f9c18da6e3b85fe0c47a12a2191c61426c2b19 quay.io/prometheus/alertmanager:v0.25.0 2024-09-10T18:55:20.140 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78762]: 2024-09-10 18:55:19.84291816 +0000 UTC m=+0.138365269 volume create 8dcfafd6996bee35ce22fefebac7c9d751a22f40c2883f4fe9edf7835b1a44de 2024-09-10T18:55:20.140 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:19 smithi107 podman[78762]: 2024-09-10 18:55:19.945878344 +0000 UTC m=+0.241325453 container create 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:20.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:20 smithi162 ceph-mon[33228]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-mon[29404]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-mon[40355]: pgmap v44: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:20.716 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 podman[78762]: 2024-09-10 18:55:20.248478388 +0000 UTC m=+0.543925519 container init 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:20.717 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 podman[78762]: 2024-09-10 18:55:20.255355041 +0000 UTC m=+0.550802145 container start 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:20.717 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.271Z caller=main.go:240 level=info msg="Starting Alertmanager" version="(version=0.25.0, branch=HEAD, revision=258fab7cdd551f2cf251ed0348f0ad7289aee789)" 2024-09-10T18:55:20.717 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.271Z caller=main.go:241 level=info build_context="(go=go1.19.4, user=root@abe866dd5717, date=20221222-14:51:36)" 2024-09-10T18:55:20.717 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.273Z caller=cluster.go:185 level=info component=cluster msg="setting advertise address explicitly" addr=172.21.15.107 port=9094 2024-09-10T18:55:20.717 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.274Z caller=cluster.go:681 level=info component=cluster msg="Waiting for gossip to settle..." interval=2s 2024-09-10T18:55:20.718 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.304Z caller=coordinator.go:113 level=info component=configuration msg="Loading configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:55:20.718 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.304Z caller=coordinator.go:126 level=info component=configuration msg="Completed loading of configuration file" file=/etc/alertmanager/alertmanager.yml 2024-09-10T18:55:20.718 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.308Z caller=tls_config.go:232 level=info msg="Listening on" address=[::]:9093 2024-09-10T18:55:20.718 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:20.309Z caller=tls_config.go:235 level=info msg="TLS is disabled." http2=false address=[::]:9093 2024-09-10T18:55:20.718 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 bash[78762]: 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 2024-09-10T18:55:20.719 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:20 smithi107 systemd[1]: Started Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:21.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:55:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:21.631 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:21 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.631 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:21 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.631 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:21 smithi162 ceph-mon[33228]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:55:21.631 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:21 smithi162 ceph-mon[33228]: Reconfiguring daemon prometheus.a on smithi162 2024-09-10T18:55:21.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[29404]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[29404]: Reconfiguring daemon prometheus.a on smithi162 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[40355]: Reconfiguring prometheus.a (dependencies changed)... 2024-09-10T18:55:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:21 smithi107 ceph-mon[40355]: Reconfiguring daemon prometheus.a on smithi162 2024-09-10T18:55:22.383 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:22.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:22 smithi162 ceph-mon[33228]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:22 smithi107 ceph-mon[29404]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:22.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:22 smithi107 ceph-mon[40355]: pgmap v45: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:22.716 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:22 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:22.275Z caller=cluster.go:706 level=info component=cluster msg="gossip not settled" polls=0 before=0 now=1 elapsed=2.000752779s 2024-09-10T18:55:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:24 smithi162 ceph-mon[33228]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:24 smithi107 ceph-mon[29404]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:24 smithi107 ceph-mon[40355]: pgmap v46: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:26.567 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:26 smithi162 ceph-mon[33228]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:26 smithi107 ceph-mon[29404]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:26.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:26 smithi107 ceph-mon[40355]: pgmap v47: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-mon[33228]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:28.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 systemd[1]: Stopping Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:28 smithi107 ceph-mon[29404]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:28.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:28 smithi107 ceph-mon[40355]: pgmap v48: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.773Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.774Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.774Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T18:55:29.073 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.774Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T18:55:29.074 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[64928]: ts=2024-09-10T18:55:28.774Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T18:55:29.074 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:28 smithi162 podman[68251]: 2024-09-10 18:55:28.784148601 +0000 UTC m=+0.132893225 container died 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:29.414 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68251]: 2024-09-10 18:55:29.162514562 +0000 UTC m=+0.511259194 container cleanup 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:29.414 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 bash[68251]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T18:55:29.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68269]: 2024-09-10 18:55:29.413820545 +0000 UTC m=+0.631608827 container remove 3115e271a417e6dd84c8919387373d621b78bcef66b872441fc21ab06f16b76c (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:29.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@prometheus.a.service: Deactivated successfully. 2024-09-10T18:55:29.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 systemd[1]: Stopped Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:29.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@prometheus.a.service: Consumed 1.110s CPU time. 2024-09-10T18:55:29.674 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 systemd[1]: Starting Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T18:55:30.013 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68366]: 2024-09-10 18:55:29.7564582 +0000 UTC m=+0.104193890 container create 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:30.013 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68366]: 2024-09-10 18:55:29.6689417 +0000 UTC m=+0.016677398 image pull a07b618ecd1dce142bce4c52f0e80982eaf1f14265a2415c2d35978ccaa0a464 quay.io/prometheus/prometheus:v2.43.0 2024-09-10T18:55:30.013 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68366]: 2024-09-10 18:55:29.984042519 +0000 UTC m=+0.331778210 container init 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:30.013 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:29 smithi162 podman[68366]: 2024-09-10 18:55:29.988752213 +0000 UTC m=+0.336487907 container start 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T18:55:30.398 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T18:55:30.278Z caller=cluster.go:698 level=info component=cluster msg="gossip settled; proceeding" elapsed=10.004020828s 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.012Z caller=main.go:564 level=info msg="Starting Prometheus Server" mode=server version="(version=2.43.0, branch=HEAD, revision=edfc3bcd025dd6fe296c167a14a216cab1e552ee)" 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.012Z caller=main.go:569 level=info build_context="(go=go1.19.7, platform=linux/amd64, user=root@8a0ee342e522, date=20230321-12:56:07, tags=netgo,builtinassets)" 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.012Z caller=main.go:570 level=info host_details="(Linux 5.14.0-505.el9.x86_64 #1 SMP PREEMPT_DYNAMIC Thu Sep 5 07:54:07 UTC 2024 x86_64 smithi162 (none))" 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.012Z caller=main.go:571 level=info fd_limits="(soft=1048576, hard=1048576)" 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.012Z caller=main.go:572 level=info vm_limits="(soft=unlimited, hard=unlimited)" 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.014Z caller=web.go:561 level=info component=web msg="Start listening for connections" address=:9095 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.014Z caller=main.go:1005 level=info msg="Starting TSDB ..." 2024-09-10T18:55:30.400 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.015Z caller=tls_config.go:232 level=info component=web msg="Listening on" address=[::]:9095 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.015Z caller=tls_config.go:235 level=info component=web msg="TLS is disabled." http2=false address=[::]:9095 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.017Z caller=head.go:587 level=info component=tsdb msg="Replaying on-disk memory mappable chunks if any" 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.017Z caller=head.go:658 level=info component=tsdb msg="On-disk memory mappable chunks replay completed" duration=1.649µs 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.017Z caller=head.go:664 level=info component=tsdb msg="Replaying WAL, this may take a while" 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=0 maxSegment=1 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=head.go:735 level=info component=tsdb msg="WAL segment loaded" segment=1 maxSegment=1 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=head.go:772 level=info component=tsdb msg="WAL replay completed" checkpoint_replay_duration=20.275µs wal_replay_duration=401.547µs wbl_replay_duration=130ns total_replay_duration=440.289µs 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=main.go:1026 level=info fs_type=XFS_SUPER_MAGIC 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=main.go:1029 level=info msg="TSDB started" 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.018Z caller=main.go:1209 level=info msg="Loading configuration file" filename=/etc/prometheus/prometheus.yml 2024-09-10T18:55:30.401 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.034Z caller=main.go:1246 level=info msg="Completed loading of configuration file" filename=/etc/prometheus/prometheus.yml totalDuration=15.711843ms db_storage=700ns remote_storage=1.076µs web_handler=362ns query_engine=553ns scrape=439.561µs scrape_sd=25.804µs notify=19.227µs notify_sd=6.718µs rules=14.99116ms tracing=6.06µs 2024-09-10T18:55:30.402 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.034Z caller=main.go:990 level=info msg="Server is ready to receive web requests." 2024-09-10T18:55:30.402 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T18:55:30.034Z caller=manager.go:974 level=info component="rule manager" msg="Starting rule manager..." 2024-09-10T18:55:30.402 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 bash[68366]: 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 2024-09-10T18:55:30.402 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 18:55:30 smithi162 systemd[1]: Started Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T18:55:30.402 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Bus STOPPING 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:30.403 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:30.404 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.404 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:30.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:30 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: pgmap v49: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:30.716 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:55:30.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Bus STOPPED 2024-09-10T18:55:30.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Bus STARTING 2024-09-10T18:55:30.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Serving on http://:::9283 2024-09-10T18:55:30.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Bus STARTED 2024-09-10T18:55:30.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:30] ENGINE Bus STOPPING 2024-09-10T18:55:31.216 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:55:31.217 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STOPPED 2024-09-10T18:55:31.217 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STARTING 2024-09-10T18:55:31.217 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Serving on http://:::9283 2024-09-10T18:55:31.217 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STARTED 2024-09-10T18:55:31.217 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STOPPING 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-alertmanager-api-host"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-alertmanager-api-host", "value": "http://smithi107.front.sepia.ceph.com:9093"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-prometheus-api-host"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-prometheus-api-host", "value": "http://smithi162.front.sepia.ceph.com:9095"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard get-grafana-api-url"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mon.0 -' entity='mon.' cmd=[{"prefix": "dashboard set-grafana-api-url", "value": "https://smithi162.front.sepia.ceph.com:3000"}]: dispatch 2024-09-10T18:55:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:31.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T18:55:31.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STOPPED 2024-09-10T18:55:31.717 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STARTING 2024-09-10T18:55:32.086 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Serving on http://:::9283 2024-09-10T18:55:32.087 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:31 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:18:55:31] ENGINE Bus STARTED 2024-09-10T18:55:32.402 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:32.407 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:32 smithi107 ceph-mon[29404]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:32.662 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:32 smithi162 ceph-mon[33228]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:32.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:32 smithi107 ceph-mon[40355]: pgmap v50: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:33 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:33 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:33.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:33 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:34 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: pgmap v51: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:34 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:35.418 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:35 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:35 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:35 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:55:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:36 smithi107 ceph-mon[29404]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:36.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:36 smithi107 ceph-mon[40355]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:36.770 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:36 smithi162 ceph-mon[33228]: pgmap v52: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:38.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[40355]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:38.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:38.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[29404]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:38 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:38.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:38 smithi162 ceph-mon[33228]: pgmap v53: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:38 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:38 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:39] "GET /metrics HTTP/1.1" 200 219790 "" "Prometheus/2.43.0" 2024-09-10T18:55:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:55:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:55:40.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:40 smithi162 ceph-mon[33228]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:40 smithi107 ceph-mon[29404]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:40 smithi107 ceph-mon[40355]: pgmap v54: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:41.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:55:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr:Note: switching to 'fad5356a94e6044ce1062d748daa5a0f2ed53a7d'. 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr:state without impacting any branches by switching back to a branch. 2024-09-10T18:55:41.865 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: git switch -c 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr:Or undo this operation with: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: git switch - 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr: 2024-09-10T18:55:41.866 INFO:tasks.workunit.client.0.smithi107.stderr:HEAD is now at fad5356a94e qa/upgrade: change pacific/octopus upgrade tests to use published image 2024-09-10T18:55:41.869 DEBUG:teuthology.orchestra.run.smithi107:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-10T18:55:41.906 INFO:tasks.workunit.client.0.smithi107.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-10T18:55:41.908 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:55:41.908 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-10T18:55:42.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:42.670 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-10T18:55:42.694 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:42 smithi162 ceph-mon[33228]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:42.741 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-10T18:55:42.783 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-10T18:55:42.784 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:55:42.784 INFO:tasks.workunit.client.0.smithi107.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-10T18:55:42.823 INFO:tasks.workunit.client.0.smithi107.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-10T18:55:42.826 DEBUG:teuthology.orchestra.run.smithi107:> set -ex 2024-09-10T18:55:42.826 DEBUG:teuthology.orchestra.run.smithi107:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-10T18:55:42.882 INFO:tasks.workunit:Running workunits matching rados/test.sh on client.0... 2024-09-10T18:55:42.884 INFO:tasks.workunit:Running workunit rados/test.sh... 2024-09-10T18:55:42.884 DEBUG:teuthology.orchestra.run.smithi107:workunit test rados/test.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test.sh 2024-09-10T18:55:42.948 INFO:tasks.workunit.client.0.smithi107.stderr:+ parallel=1 2024-09-10T18:55:42.948 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' '' = --serial ']' 2024-09-10T18:55:42.948 INFO:tasks.workunit.client.0.smithi107.stderr:+ color= 2024-09-10T18:55:42.948 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -t 1 ']' 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ trap cleanup EXIT ERR HUP INT QUIT 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ declare -A pids 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_aio 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_aio' 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_aio 2024-09-10T18:55:42.949 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_aio 2024-09-10T18:55:42.950 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79983 2024-09-10T18:55:42.950 INFO:tasks.workunit.client.0.smithi107.stdout:test api_aio on pid 79983 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_aio on pid 79983' 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=79983 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio 2>&1 | tee ceph_test_rados_api_aio.log | sed "s/^/ api_aio: /"' 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_aio_pp 2024-09-10T18:55:42.953 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_aio_pp' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_aio_pp 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.954 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.955 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:42.956 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_aio_pp 2024-09-10T18:55:42.956 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79990 2024-09-10T18:55:42.956 INFO:tasks.workunit.client.0.smithi107.stdout:test api_aio_pp on pid 79990 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_aio_pp on pid 79990' 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=79990 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_aio_pp 2>&1 | tee ceph_test_rados_api_aio_pp.log | sed "s/^/ api_aio_pp: /"' 2024-09-10T18:55:42.959 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_io 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_io' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_io 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:42.960 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:42.961 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:42.962 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_io 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79999 2024-09-10T18:55:42.963 INFO:tasks.workunit.client.0.smithi107.stdout:test api_io on pid 79999 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_io on pid 79999' 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=79999 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io 2>&1 | tee ceph_test_rados_api_io.log | sed "s/^/ api_io: /"' 2024-09-10T18:55:42.967 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_io_pp 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_io_pp' 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_io_pp 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:42.968 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:42.969 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:42.970 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.971 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_io_pp 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80010 2024-09-10T18:55:42.972 INFO:tasks.workunit.client.0.smithi107.stdout:test api_io_pp on pid 80010 2024-09-10T18:55:42.977 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_io_pp on pid 80010' 2024-09-10T18:55:42.977 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80010 2024-09-10T18:55:42.977 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.977 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_io_pp 2>&1 | tee ceph_test_rados_api_io_pp.log | sed "s/^/ api_io_pp: /"' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_asio 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_asio' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_asio 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:42.978 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.979 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:42.980 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:42.981 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.982 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.983 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_asio 2024-09-10T18:55:42.984 INFO:tasks.workunit.client.0.smithi107.stdout:test api_asio on pid 80025 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80025 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_asio on pid 80025' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80025 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_asio 2>&1 | tee ceph_test_rados_api_asio.log | sed "s/^/ api_asio: /"' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:42.990 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_list 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_list' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_list 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.991 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:42.992 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:42.993 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:42.994 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:42.995 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:42.996 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:42.997 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/79983/exe 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_list 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:42.998 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80043 2024-09-10T18:55:42.999 INFO:tasks.workunit.client.0.smithi107.stdout:test api_list on pid 80043 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_list on pid 80043' 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80043 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_list 2>&1 | tee ceph_test_rados_api_list.log | sed "s/^/ api_list: /"' 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_lock 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_lock' 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_lock 2024-09-10T18:55:43.000 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.001 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:42 smithi107 ceph-mon[40355]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:43.001 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:42 smithi107 ceph-mon[29404]: pgmap v55: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.002 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.003 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.004 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.005 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.006 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.007 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_aio 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.008 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_aio: /' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/79990/exe 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.009 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_lock 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.010 INFO:tasks.workunit.client.0.smithi107.stdout:test api_lock on pid 80063 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80063 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_lock on pid 80063' 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80063 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_aio.log 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_lock_pp 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock 2>&1 | tee ceph_test_rados_api_lock.log | sed "s/^/ api_lock: /"' 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_lock_pp' 2024-09-10T18:55:43.014 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_lock_pp 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.015 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.016 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_aio_pp 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_aio_pp: /' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.017 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.018 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.019 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.020 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/79999/exe 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.021 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.022 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_aio_pp.log 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.023 INFO:tasks.workunit.client.0.smithi107.stdout:test api_lock_pp on pid 80081 2024-09-10T18:55:43.030 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.030 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_lock_pp 2024-09-10T18:55:43.030 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80081 2024-09-10T18:55:43.030 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_lock_pp on pid 80081' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80081 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_lock_pp 2>&1 | tee ceph_test_rados_api_lock_pp.log | sed "s/^/ api_lock_pp: /"' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.031 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_misc 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_misc' 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.032 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_io 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.033 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.034 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.035 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.036 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.037 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.038 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.039 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_misc 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_io.log 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80010/exe 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.040 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_io: /' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.041 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.042 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.043 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.044 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.045 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_io_pp 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.046 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80025/exe 2024-09-10T18:55:43.047 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_io_pp.log 2024-09-10T18:55:43.047 INFO:tasks.workunit.client.0.smithi107.stdout:test api_misc on pid 80115 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_misc 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80115 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.050 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_misc on pid 80115' 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80115 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc 2>&1 | tee ceph_test_rados_api_misc.log | sed "s/^/ api_misc: /"' 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.051 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_misc_pp 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_misc_pp' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_asio.log 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.052 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_asio: /' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_io_pp: /' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_asio 2024-09-10T18:55:43.053 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.054 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.055 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80043/exe 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.056 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.057 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.058 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_misc_pp 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.059 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_list 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_list.log 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.060 INFO:tasks.workunit.client.0.smithi107.stdout:test api_misc_pp on pid 80139 2024-09-10T18:55:43.063 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_misc_pp 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80139 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_misc_pp on pid 80139' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80139 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_misc_pp 2>&1 | tee ceph_test_rados_api_misc_pp.log | sed "s/^/ api_misc_pp: /"' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.064 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.065 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.066 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_list: /' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.067 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.068 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80063/exe 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80081/exe 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_tier_pp 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_tier_pp' 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.069 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_lock_pp 2024-09-10T18:55:43.070 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_lock.log 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_lock 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_lock_pp: /' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_lock: /' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.071 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.072 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.073 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_lock_pp.log 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_tier_pp 2024-09-10T18:55:43.074 INFO:tasks.workunit.client.0.smithi107.stdout:test api_tier_pp on pid 80181 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_tier_pp 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80181 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_tier_pp on pid 80181' 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80181 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.076 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_tier_pp 2>&1 | tee ceph_test_rados_api_tier_pp.log | sed "s/^/ api_tier_pp: /"' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.077 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_pool 2024-09-10T18:55:43.078 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_pool' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.079 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.080 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_pool 2024-09-10T18:55:43.081 INFO:tasks.workunit.client.0.smithi107.stdout:test api_pool on pid 80203 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_pool 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80203 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_pool on pid 80203' 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80203 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.087 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_pool 2>&1 | tee ceph_test_rados_api_pool.log | sed "s/^/ api_pool: /"' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_snapshots 2024-09-10T18:55:43.088 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_snapshots' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.089 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.090 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.091 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.092 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.093 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.094 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_snapshots 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.095 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.096 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_snapshots 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80265 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_snapshots on pid 80265' 2024-09-10T18:55:43.097 INFO:tasks.workunit.client.0.smithi107.stdout:test api_snapshots on pid 80265 2024-09-10T18:55:43.103 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80265 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots 2>&1 | tee ceph_test_rados_api_snapshots.log | sed "s/^/ api_snapshots: /"' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.104 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.105 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.106 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.107 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.108 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.109 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80181/exe 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.110 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.111 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.112 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_snapshots_pp 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.113 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_snapshots_pp' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.114 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.115 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.116 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_snapshots_pp 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_tier_pp: /' 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_tier_pp.log 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_tier_pp 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80139/exe 2024-09-10T18:55:43.117 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.118 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_misc_pp 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_misc_pp: /' 2024-09-10T18:55:43.119 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.120 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.121 INFO:tasks.workunit.client.0.smithi107.stdout:test api_snapshots_pp on pid 80314 2024-09-10T18:55:43.124 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_snapshots_pp 2024-09-10T18:55:43.124 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80314 2024-09-10T18:55:43.124 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_snapshots_pp on pid 80314' 2024-09-10T18:55:43.124 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80314 2024-09-10T18:55:43.124 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_snapshots_pp 2>&1 | tee ceph_test_rados_api_snapshots_pp.log | sed "s/^/ api_snapshots_pp: /"' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.125 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_stat 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_stat' 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80115/exe 2024-09-10T18:55:43.126 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.127 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80203/exe 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_misc_pp.log 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.128 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_stat 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.129 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_misc 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_misc: /' 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_misc.log 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_pool.log 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_pool 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_stat 2024-09-10T18:55:43.130 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80342 2024-09-10T18:55:43.131 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_stat on pid 80342' 2024-09-10T18:55:43.131 INFO:tasks.workunit.client.0.smithi107.stdout:test api_stat on pid 80342 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80342 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_stat_pp 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_stat_pp' 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_stat_pp 2024-09-10T18:55:43.132 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.133 INFO:tasks.workunit.client.0.smithi107.stdout:test api_stat_pp on pid 80352 2024-09-10T18:55:43.133 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_stat_pp 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80352 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_stat_pp on pid 80352' 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80352 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.134 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_watch_notify 2024-09-10T18:55:43.135 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_watch_notify' 2024-09-10T18:55:43.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_pool: /' 2024-09-10T18:55:43.136 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.136 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_watch_notify 2024-09-10T18:55:43.136 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat 2>&1 | tee ceph_test_rados_api_stat.log | sed "s/^/ api_stat: /"' 2024-09-10T18:55:43.136 INFO:tasks.workunit.client.0.smithi107.stdout:test api_watch_notify on pid 80362 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_watch_notify 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80362 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_watch_notify on pid 80362' 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80362 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.143 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify 2>&1 | tee ceph_test_rados_api_watch_notify.log | sed "s/^/ api_watch_notify: /"' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_watch_notify_pp 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_watch_notify_pp' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.144 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.145 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_watch_notify_pp 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_stat_pp 2>&1 | tee ceph_test_rados_api_stat_pp.log | sed "s/^/ api_stat_pp: /"' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.146 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.147 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.148 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.149 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.150 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.151 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80265/exe 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.152 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.153 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.154 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_snapshots.log 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_snapshots 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_snapshots: /' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.155 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.156 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_watch_notify_pp 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.157 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80397 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.158 INFO:tasks.workunit.client.0.smithi107.stdout:test api_watch_notify_pp on pid 80397 2024-09-10T18:55:43.161 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_watch_notify_pp on pid 80397' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80397 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_watch_notify_pp 2>&1 | tee ceph_test_rados_api_watch_notify_pp.log | sed "s/^/ api_watch_notify_pp: /"' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_cmd 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_cmd' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_cmd 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.162 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.163 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.164 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.165 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.166 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.167 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.167 INFO:tasks.workunit.client.0.smithi107.stdout:test api_cmd on pid 80418 2024-09-10T18:55:43.173 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_cmd 2024-09-10T18:55:43.173 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80418 2024-09-10T18:55:43.173 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_cmd on pid 80418' 2024-09-10T18:55:43.173 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80418 2024-09-10T18:55:43.173 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.174 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd 2>&1 | tee ceph_test_rados_api_cmd.log | sed "s/^/ api_cmd: /"' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.175 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.176 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_cmd_pp 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.177 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.178 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_cmd_pp' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_cmd_pp 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.179 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.180 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.181 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.182 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.183 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.184 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_cmd_pp 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.185 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80471 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_cmd_pp on pid 80471' 2024-09-10T18:55:43.186 INFO:tasks.workunit.client.0.smithi107.stdout:test api_cmd_pp on pid 80471 2024-09-10T18:55:43.188 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80471 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_service 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_service' 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80342/exe 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.189 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.190 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cmd_pp 2>&1 | tee ceph_test_rados_api_cmd_pp.log | sed "s/^/ api_cmd_pp: /"' 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.191 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.192 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.193 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.194 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_stat 2024-09-10T18:55:43.195 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_stat.log 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80314/exe 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_stat: /' 2024-09-10T18:55:43.196 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_service 2024-09-10T18:55:43.197 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.197 INFO:tasks.workunit.client.0.smithi107.stdout:test api_service on pid 80500 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_service 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80500 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_service on pid 80500' 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80500 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service 2>&1 | tee ceph_test_rados_api_service.log | sed "s/^/ api_service: /"' 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.202 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.203 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80362/exe 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_snapshots_pp: /' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.204 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.205 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_snapshots_pp.log 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_service_pp 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_snapshots_pp 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_service_pp' 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.206 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.207 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.208 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.209 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_service_pp 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_watch_notify 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_watch_notify.log 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.210 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.211 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.212 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.213 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.214 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_watch_notify: /' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.215 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.216 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_service_pp 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80558 2024-09-10T18:55:43.217 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_service_pp on pid 80558' 2024-09-10T18:55:43.218 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80558 2024-09-10T18:55:43.218 INFO:tasks.workunit.client.0.smithi107.stdout:test api_service_pp on pid 80558 2024-09-10T18:55:43.222 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.222 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.222 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_service_pp 2>&1 | tee ceph_test_rados_api_service_pp.log | sed "s/^/ api_service_pp: /"' 2024-09-10T18:55:43.222 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80352/exe 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_c_write_operations 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_c_write_operations' 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80397/exe 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.223 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.224 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_c_write_operations 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_watch_notify_pp: /' 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_watch_notify_pp.log 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.225 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.226 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_stat_pp 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.227 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.228 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.229 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.229 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.229 INFO:tasks.workunit.client.0.smithi107.stdout:test api_c_write_operations on pid 80579 2024-09-10T18:55:43.234 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_c_write_operations 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80579 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_c_write_operations on pid 80579' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80579 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_write_operations 2>&1 | tee ceph_test_rados_api_c_write_operations.log | sed "s/^/ api_c_write_operations: /"' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.235 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_c_read_operations 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_c_read_operations' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.236 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_stat_pp.log 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_watch_notify_pp 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_stat_pp: /' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_c_read_operations 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.237 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.238 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.239 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.240 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.241 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.242 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.243 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.244 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.245 INFO:tasks.workunit.client.0.smithi107.stdout:test api_c_read_operations on pid 80610 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_c_read_operations 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80610 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_c_read_operations on pid 80610' 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80610 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.246 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s api_cls_remote_reads 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' api_cls_remote_reads' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80418/exe 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_c_read_operations 2>&1 | tee ceph_test_rados_api_c_read_operations.log | sed "s/^/ api_c_read_operations: /"' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.247 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.248 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo api_cls_remote_reads 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.249 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.250 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_cmd 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.251 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.252 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.253 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80471/exe 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.254 INFO:tasks.workunit.client.0.smithi107.stdout:test api_cls_remote_reads on pid 80641 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.256 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=api_cls_remote_reads 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80641 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test api_cls_remote_reads on pid 80641' 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80641 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_api_cls_remote_reads 2>&1 | tee ceph_test_rados_api_cls_remote_reads.log | sed "s/^/ api_cls_remote_reads: /"' 2024-09-10T18:55:43.257 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s list_parallel 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' list_parallel' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.258 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.259 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_cmd_pp 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80500/exe 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_cmd_pp.log 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_cmd_pp: /' 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo list_parallel 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.260 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.261 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.261 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.261 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.261 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_service 2024-09-10T18:55:43.261 INFO:tasks.workunit.client.0.smithi107.stdout:test list_parallel on pid 80663 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=list_parallel 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80663 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test list_parallel on pid 80663' 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80663 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s open_pools_parallel 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' open_pools_parallel' 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_list_parallel 2>&1 | tee ceph_test_rados_list_parallel.log | sed "s/^/ list_parallel: /"' 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_cmd.log 2024-09-10T18:55:43.267 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.268 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.269 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.270 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_cmd: /' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.271 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_service.log 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.272 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.273 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.274 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_service: /' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.275 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.276 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.277 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo open_pools_parallel 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.278 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.279 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.280 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.281 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.282 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.283 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=open_pools_parallel 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80724 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test open_pools_parallel on pid 80724' 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80724 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in api_aio api_aio_pp api_io api_io_pp api_asio api_list api_lock api_lock_pp api_misc api_misc_pp api_tier_pp api_pool api_snapshots api_snapshots_pp api_stat api_stat_pp api_watch_notify api_watch_notify_pp api_cmd api_cmd_pp api_service api_service_pp api_c_write_operations api_c_read_operations api_cls_remote_reads list_parallel open_pools_parallel delete_pools_parallel 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.284 INFO:tasks.workunit.client.0.smithi107.stdout:test open_pools_parallel on pid 80724 2024-09-10T18:55:43.286 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++ printf %25s delete_pools_parallel 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+ r=' delete_pools_parallel' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.287 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:++ awk '{print $1}' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_open_pools_parallel 2>&1 | tee ceph_test_rados_open_pools_parallel.log | sed "s/^/ open_pools_parallel: /"' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:++ echo delete_pools_parallel 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.288 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.289 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.290 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.291 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.292 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.293 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.293 INFO:tasks.workunit.client.0.smithi107.stdout:test delete_pools_parallel on pid 80757 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ ff=delete_pools_parallel 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80757 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo 'test delete_pools_parallel on pid 80757' 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ pids[$f]=80757 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ ret=0 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 1 -eq 1 ']' 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T18:55:43.302 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80181 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80181 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80558/exe 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+ bash -o pipefail -exc 'ceph_test_rados_delete_pools_parallel 2>&1 | tee ceph_test_rados_delete_pools_parallel.log | sed "s/^/ delete_pools_parallel: /"' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.303 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.304 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.305 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80579/exe 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.306 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_service_pp.log 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -f /etc/bashrc ']' 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+ . /etc/bashrc 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -z '' ']' 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:++ BASHRCSOURCED=Y 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:++ shopt -q login_shell 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_service_pp 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.307 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80610/exe 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.308 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.309 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:+++ umask 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' 0022 -eq 0 ']' 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ SHELL=/bin/bash 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/bash_completion.sh ']' 2024-09-10T18:55:43.310 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/bash_completion.sh 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' 'x5.1.8(1)-release' '!=' x -a x '!=' x -a x = x ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_c_write_operations: /' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/cobbler.sh ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/cobbler.sh 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:+++ COBBLER_SERVER=172.21.0.11 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorgrep.sh ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.311 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorgrep.sh 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_c_write_operations.log 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_service_pp: /' 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.312 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_c_write_operations 2024-09-10T18:55:43.313 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.314 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80641/exe 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.315 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_cls_remote_reads 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_cls_remote_reads: /' 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_cls_remote_reads.log 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.316 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_api_c_read_operations 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_api_c_read_operations.log 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ api_c_read_operations: /' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.317 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'grep=grep --color=auto' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'egrep=egrep --color=auto' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'fgrep=fgrep --color=auto' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorls.sh ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorls.sh 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' '!' -t 0 ']' 2024-09-10T18:55:43.318 INFO:tasks.workunit.client.0.smithi107.stderr:+++ return 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorsysstat.sh ']' 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorsysstat.sh 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export S_COLORS=auto 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:+++ S_COLORS=auto 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorxzgrep.sh ']' 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorxzgrep.sh 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.319 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.320 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80663/exe 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.321 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.322 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ list_parallel: /' 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_list_parallel.log 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_list_parallel 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzgrep=xzgrep --color=auto' 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzegrep=xzegrep --color=auto' 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'xzfgrep=xzfgrep --color=auto' 2024-09-10T18:55:43.323 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/colorzgrep.sh ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/colorzgrep.sh 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /usr/libexec/grepconf.sh ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ /usr/libexec/grepconf.sh -c 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.324 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.325 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80724/exe 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.326 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_open_pools_parallel.log 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zgrep=zgrep --color=auto' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zfgrep=zfgrep --color=auto' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+++ alias 'zegrep=zegrep --color=auto' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/debuginfod.sh ']' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/debuginfod.sh 2024-09-10T18:55:43.327 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z 'https://debuginfod.centos.org/ ' ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/gawk.sh ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/gawk.sh 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_open_pools_parallel 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ open_pools_parallel: /' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/lang.sh ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/lang.sh 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/locale 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.328 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG_backup=en_US.UTF-8 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /etc/locale.conf ']' 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -x /usr/bin/sed ']' 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:++++ /usr/bin/sed -r -e 's/^[[:blank:]]*([[:upper:]_]+)=([[:print:][:digit:]\._-]+|"[[:print:][:digit:]\._-]+")/export \1=\2/;t;d' /etc/locale.conf 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ eval export 'LANG="en_US.UTF-8"' 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:++++ export LANG=en_US.UTF-8 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:++++ LANG=en_US.UTF-8 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ for config in /etc/locale.conf "${HOME}/.i18n" 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -f /home/ubuntu/.i18n ']' 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ LANG=en_US.UTF-8 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ unset LANG_backup config 2024-09-10T18:55:43.329 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n '' ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -n en_US.UTF-8 ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' dumb = linux ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/less.sh ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/less.sh 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:+++ '[' -z '||/usr/bin/lesspipe.sh %s' ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ for i in /etc/profile.d/*.sh 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' -r /etc/profile.d/which2.sh ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ '[' '' ']' 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:++ . /etc/profile.d/which2.sh 2024-09-10T18:55:43.330 INFO:tasks.workunit.client.0.smithi107.stderr:+++ case "$(basename $(readlink /proc/$$/exe))" in 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+++++ readlink /proc/80757/exe 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:++++ basename /usr/bin/bash 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_declare='declare -f' 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+++ which_opt=-f 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export which_declare 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+++ export -f which 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset i 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:++ unset -f pathmunge 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+ [[ /home/ubuntu/.local/bin:/home/ubuntu/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/usr/sbin =~ /home/ubuntu/\.local/bin:/home/ubuntu/bin: ]] 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+ export PATH 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' -d /home/ubuntu/.bashrc.d ']' 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+ unset rc 2024-09-10T18:55:43.331 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph_test_rados_delete_pools_parallel 2024-09-10T18:55:43.332 INFO:tasks.workunit.client.0.smithi107.stderr:+ tee ceph_test_rados_delete_pools_parallel.log 2024-09-10T18:55:43.332 INFO:tasks.workunit.client.0.smithi107.stderr:+ sed 's/^/ delete_pools_parallel: /' 2024-09-10T18:55:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3881548061' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:55:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3881548061' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:55:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3881548061' entity='client.admin' cmd=[{"prefix":"quorum_status"}]: dispatch 2024-09-10T18:55:46.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:46.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/904498033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1510483992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4203571855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4130390675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1896074661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1307207666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:46.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/904498033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1510483992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4203571855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4130390675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1896074661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1307207666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:55:46.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: pgmap v56: 132 pgs: 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/904498033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: osdmap e64: 8 total, 8 up, 8 in 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1510483992' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4203571855' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4130390675' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1896074661' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1307207666' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: osdmap e65: 8 total, 8 up, 8 in 2024-09-10T18:55:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.066 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: pgmap v59: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.067 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: pgmap v59: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.068 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: pgmap v59: 356 pgs: 224 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:49] "GET /metrics HTTP/1.1" 200 265705 "" "Prometheus/2.43.0" 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [==========] Running 12 tests from 1 test suite. 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] Global test environment set-up. 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] 12 tests from AsioRados 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadCallback 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadCallback (3 ms) 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadFuture 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadFuture (1 ms) 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadYield 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadYield (2 ms) 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteCallback 2024-09-10T18:55:50.049 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteCallback (5 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteFuture 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteFuture (15 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteYield 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteYield (2 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationCallback 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationCallback (4 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationFuture 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationFuture (3 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncReadOperationYield 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncReadOperationYield (2 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationCallback 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationCallback (3 ms) 2024-09-10T18:55:50.050 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationFuture 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationFuture (7 ms) 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ RUN ] AsioRados.AsyncWriteOperationYield 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ OK ] AsioRados.AsyncWriteOperationYield (2 ms) 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] 12 tests from AsioRados (50 ms total) 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [----------] Global test environment tear-down 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [==========] 12 tests from 1 test suite ran. (7017 ms total) 2024-09-10T18:55:50.051 INFO:tasks.workunit.client.0.smithi107.stdout: api_asio: [ PASSED ] 12 tests. 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1189788528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1571458734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: pgmap v61: 516 pgs: 357 creating+peering, 20 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:55:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1023951111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2789791168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/753702941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3260095620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3030394972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3670056916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3324219001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1156074424' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2548067167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: pgmap v63: 900 pgs: 357 creating+peering, 404 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/209661105' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:50.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1189788528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1571458734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: pgmap v61: 516 pgs: 357 creating+peering, 20 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1023951111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2789791168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/753702941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3260095620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3030394972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3670056916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3324219001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1156074424' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2548067167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: pgmap v63: 900 pgs: 357 creating+peering, 404 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/209661105' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2596146385' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIo_smithi107-80087-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1189788528' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1571458734' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: pgmap v61: 516 pgs: 357 creating+peering, 20 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: osdmap e66: 8 total, 8 up, 8 in 2024-09-10T18:55:50.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: Health check failed: 7 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBigPP_smithi107-80070-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLockPP_smithi107-80149-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotify_smithi107-80536-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "TooBig_smithi107-80053-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.221 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1023951111' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosIoPP_smithi107-80108-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2789791168' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/753702941' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3260095620' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3030394972' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3670056916' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3324219001' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1156074424' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2548067167' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: osdmap e67: 8 total, 8 up, 8 in 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: pgmap v63: 900 pgs: 357 creating+peering, 404 unknown, 139 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 933 B/s rd, 0 op/s 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.222 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:50.223 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/209661105' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [==========] Running 11 tests from 3 test suites. 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] Global test environment set-up. 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 7 tests from LibRadosList 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjects 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjects (301 ms) 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsZeroInName 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsZeroInName (30 ms) 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsNS 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:55:50.359 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo2 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo3 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo4 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo5 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo6,foo7 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo7 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo6 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo4 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo5 2024-09-10T18:55:50.360 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo7 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo6 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo1 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo1 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo2 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo3 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsNS (83 ms) 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsStart 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 1 0 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 10 0 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 13 0 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 7 0 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 14 0 2024-09-10T18:55:50.361 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 0 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 15 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 11 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 5 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 8 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 6 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 3 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 4 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 12 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 9 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2 0 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsStart (56 ms) 2024-09-10T18:55:50.362 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.ListObjectsCursor 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: x cursor=MIN 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=1 cursor=14:02547ec2:::1:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=10 cursor=14:52ea6a34:::10:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=13 cursor=14:566253c9:::13:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=7 cursor=14:5c6b0b28:::7:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=14 cursor=14:62a1935d:::14:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=0 cursor=14:6cac518f:::0:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=15 cursor=14:863748b0:::15:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=11 cursor=14:89d3ae78:::11:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=5 cursor=14:b29083e3:::5:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=8 cursor=14:bd63b0f1:::8:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=6 cursor=14:c4fdafeb:::6:head 2024-09-10T18:55:50.363 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=3 cursor=14:cfc208b3:::3:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=4 cursor=14:d83876eb:::4:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=12 cursor=14:de5d7c5f:::12:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=9 cursor=14:e960b815:::9:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > oid=2 cursor=14:f905c69b:::2:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: FIRST> seek to MIN oid=1 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=1 cursor=14:02547ec2:::1:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=10 cursor=14:52ea6a34:::10:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=13 cursor=14:566253c9:::13:head 2024-09-10T18:55:50.364 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-10T18:55:50.365 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-10T18:55:50.365 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=7 cursor=14:5c6b0b28:::7:head 2024-09-10T18:55:50.365 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-10T18:55:50.365 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=14 cursor=14:62a1935d:::14:head 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=0 cursor=14:6cac518f:::0:head 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-10T18:55:50.408 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=15 cursor=14:863748b0:::15:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=11 cursor=14:89d3ae78:::11:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=5 cursor=14:b29083e3:::5:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=8 cursor=14:bd63b0f1:::8:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-10T18:55:50.409 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=6 cursor=14:c4fdafeb:::6:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=3 cursor=14:cfc208b3:::3:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=4 cursor=14:d83876eb:::4:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=12 cursor=14:de5d7c5f:::12:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-10T18:55:50.410 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=9 cursor=14:e960b815:::9:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : oid=2 cursor=14:f905c69b:::2:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:cfc208b3:::3:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:cfc208b3:::3:head expected=14:cfc208b3:::3:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:cfc208b3:::3:head -> 3 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=3 expected=3 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:de5d7c5f:::12:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:de5d7c5f:::12:head expected=14:de5d7c5f:::12:head 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:de5d7c5f:::12:head -> 12 2024-09-10T18:55:50.411 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=12 expected=12 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:d83876eb:::4:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:d83876eb:::4:head expected=14:d83876eb:::4:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:d83876eb:::4:head -> 4 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=4 expected=4 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:f905c69b:::2:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:f905c69b:::2:head expected=14:f905c69b:::2:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:f905c69b:::2:head -> 2 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=2 expected=2 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:e960b815:::9:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:e960b815:::9:head expected=14:e960b815:::9:head 2024-09-10T18:55:50.412 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:e960b815:::9:head -> 9 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=9 expected=9 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:863748b0:::15:head 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:863748b0:::15:head expected=14:863748b0:::15:head 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:863748b0:::15:head -> 15 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=15 expected=15 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:6cac518f:::0:head 2024-09-10T18:55:50.413 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:6cac518f:::0:head expected=14:6cac518f:::0:head 2024-09-10T18:55:50.414 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:6cac518f:::0:head -> 0 2024-09-10T18:55:50.414 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=0 expected=0 2024-09-10T18:55:50.414 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:62a1935d:::14:head 2024-09-10T18:55:50.414 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:62a1935d:::14:head expected=14:62a1935d:::14:head 2024-09-10T18:55:50.415 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:62a1935d:::14:head -> 14 2024-09-10T18:55:51.162 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : ent list_parallel: process_1_[80880]: starting. 2024-09-10T18:55:51.162 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: creating pool ceph_test_rados_list_parallel.smithi107-80835 2024-09-10T18:55:51.162 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: created object 0... 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: created object 25... 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: created object 49... 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: finishing. 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_1_[80880]: shutting down. 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: starting. 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: listing objects. 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: listed object 0... 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: listed object 25... 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: saw 50 objects 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_2_[80883]: shutting down. 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.163 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: starting. 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: creating pool ceph_test_rados_list_parallel.smithi107-80835 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: created object 0... 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: created object 25... 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: created object 49... 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: finishing. 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_3_[81609]: shutting down. 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: starting. 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: listing objects. 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: listed object 0... 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: listed object 25... 2024-09-10T18:55:51.164 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: saw 42 objects 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_4_[81610]: shutting down. 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: starting. 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: removed 25 objects... 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: removed half of the objects 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: removed 50 objects... 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: removed 50 objects 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_5_[81611]: shutting down. 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: starting. 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: creating pool ceph_test_rados_list_parallel.smithi107-80835 2024-09-10T18:55:51.165 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: created object 0... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: created object 25... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: created object 49... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: finishing. 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_6_[81667]: shutting down. 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: starting. 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: listing objects. 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: listed object 0... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: listed object 25... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: listed object 50... 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: saw 57 objects 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_7_[81668]: shutting down. 2024-09-10T18:55:51.166 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: starting. 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: added 25 objects... 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: added half of the objects 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: added 50 objects... 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: added 50 objects 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_8_[81669]: shutting down. 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.167 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:55:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:51.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsPP_smithi107-80513-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosList_smithi107-80132-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosTierPP_smithi107-80290-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80651-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStat_smithi107-80486-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosStatPP_smithi107-80566-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80804-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "CReadOpsTest_smithi107-80807-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80630-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosLock_smithi107-80150-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshots_smithi107-80386-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: osdmap e68: 8 total, 8 up, 8 in 2024-09-10T18:55:51.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: starting. 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: creating pool ceph_test_rados_list_parallel.smithi107-80835 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: created object 0... 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: created object 25... 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: created object 49... 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: finishing. 2024-09-10T18:55:51.780 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_9_[81717]: shutting down. 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: starting. 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: listing objects. 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: listed object 0... 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: listed object 25... 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: listed object 50... 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: listed object 75... 2024-09-10T18:55:51.781 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: saw 92 objects 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_10_[81718]: shutting down. 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: starting. 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: removed 25 objects... 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: removed half of the objects 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: removed 50 objects... 2024-09-10T18:55:51.782 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: removed 50 objects 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_13_[81721]: shutting down. 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: starting. 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: added 25 objects... 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: added half of the objects 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: added 50 objects... 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: added 50 objects 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_11_[81719]: shutting down. 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.783 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: starting. 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: added 25 objects... 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: added half of the objects 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: added 50 objects... 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: added 50 objects 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_12_[81720]: shutting down. 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: starting. 2024-09-10T18:55:51.784 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: creating pool ceph_test_rados_list_parallel.smithi107-80835 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: created object 0... 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: created object 25... 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: created object 49... 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: finishing. 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_14_[81816]: shutting down. 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: starting. 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listing objects. 2024-09-10T18:55:51.785 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 0... 2024-09-10T18:55:52.076 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 2 api_cmd_pp: Running main() from gmock_main.cc 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [==========] Running 3 tests from 1 test suite. 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] Global test environment set-up. 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.MonDescribePP 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.MonDescribePP (40 ms) 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.OSDCmdPP 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.OSDCmdPP (44 ms) 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ RUN ] LibRadosCmd.PGCmdPP 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ OK ] LibRadosCmd.PGCmdPP (8821 ms) 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] 3 tests from LibRadosCmd (8905 ms total) 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: 2024-09-10T18:55:52.077 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [----------] Global test environment tear-down 2024-09-10T18:55:52.078 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [==========] 3 tests from 1 test suite ran. (8906 ms total) 2024-09-10T18:55:52.078 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd_pp: [ PASSED ] 3 tests. 2024-09-10T18:55:52.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: Running main() from gmock_main.cc 2024-09-10T18:55:52.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [==========] Running 16 tests from 1 test suite. 2024-09-10T18:55:52.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] Global test environment set-up. 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.NewDelete 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.NewDelete (0 ms) 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.SetOpFlags 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.SetOpFlags (262 ms) 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertExists 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertExists (10 ms) 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.AssertVersion 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.AssertVersion (7 ms) 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpXattr 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpXattr (24 ms) 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Read 2024-09-10T18:55:52.084 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Read (9 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Checksum 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Checksum (6 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.RWOrderedRead 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.RWOrderedRead (5 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ShortRead 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ShortRead (6 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Exec 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Exec (6 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.ExecUserBuf 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.ExecUserBuf (5 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Stat 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Stat (4 ms) 2024-09-10T18:55:52.085 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.Omap 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.Omap (13 ms) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.OmapNuls 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.OmapNuls (11 ms) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.GetXattrs 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.GetXattrs (9 ms) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ RUN ] CReadOpsTest.CmpExt 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ OK ] CReadOpsTest.CmpExt (3 ms) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] 16 tests from CReadOpsTest (381 ms total) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [----------] Global test environment tear-down 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [==========] 16 tests from 1 test suite ran. (8869 ms total) 2024-09-10T18:55:52.086 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_read_operations: [ PASSED ] 16 tests. 2024-09-10T18:55:52.091 INFO:tasks.workunit.client.0.smithi107.stdout:5... 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 50... 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 75... 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 100... 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: listed object 125... 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: saw 150 objects 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_15_[81817]: shutting down. 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: starting. 2024-09-10T18:55:52.092 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: added 25 objects... 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: added half of the objects 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: added 50 objects... 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: added 50 objects 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: process_16_[81818]: shutting down. 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******************************* 2024-09-10T18:55:52.093 INFO:tasks.workunit.client.0.smithi107.stdout: list_parallel: ******* SUCCESS ********** 2024-09-10T18:55:52.128 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [==========] Running 1 test from 1 test suite. 2024-09-10T18:55:52.128 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [----------] Global test environment set-up. 2024-09-10T18:55:52.128 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads 2024-09-10T18:55:52.128 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [ RUN ] ClsTestRemoteReads.TestGather 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [ OK ] ClsTestRemoteReads.TestGather (8903 ms) 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [----------] 1 test from ClsTestRemoteReads (8903 ms total) 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [----------] Global test environment tear-down 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [==========] 1 test from 1 test suite ran. (8904 ms total) 2024-09-10T18:55:52.129 INFO:tasks.workunit.client.0.smithi107.stdout: api_cls_remote_reads: [ PASSED ] 1 test. 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: Running main() from gmock_main.cc 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] Global test environment set-up. 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] 4 tests from LibRadosCmd 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.MonDescribe 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.MonDescribe (48 ms) 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.OSDCmd 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.OSDCmd (37 ms) 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.PGCmd 2024-09-10T18:55:52.191 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.PGCmd (8832 ms) 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ RUN ] LibRadosCmd.WatchLog 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.075641+0000 mon.c [INF] from='client.? 172.21.15.107:0/3590394909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.355717+0000 mon.c [INF] from='client.? 172.21.15.107:0/3599428758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.355761+0000 mon.b [INF] from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.659030+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.659282+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.659592+0000 mon.c [INF] from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.661109+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.661355+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.662074+0000 mon.c [INF] from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.662600+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.663097+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.192 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.663150+0000 mon.b [INF] from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:55:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: pgmap v65: 772 pgs: 204 creating+peering, 436 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3590394909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1581598374' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3599428758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:52.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: pgmap v65: 772 pgs: 204 creating+peering, 436 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:52.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3590394909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1581598374' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:52.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3599428758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: pgmap v65: 772 pgs: 204 creating+peering, 436 unknown, 132 active+clean; 582 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3590394909' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1581598374' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: osdmap e69: 8 total, 8 up, 8 in 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3599428758' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:52.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:52.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:53.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.664326+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_ api_service: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:55:53.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] Global test environment set-up. 2024-09-10T18:55:53.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] 4 tests from LibRadosService 2024-09-10T18:55:53.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.RegisterEarly 2024-09-10T18:55:53.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.RegisterEarly (5050 ms) 2024-09-10T18:55:53.238 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.RegisterLate 2024-09-10T18:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.RegisterLate (593 ms) 2024-09-10T18:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.StatusFormat 2024-09-10T18:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: cluster: 2024-09-10T18:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: id: d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:55:53.239 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 7 pool(s) do not have an application enabled 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: services: 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mgr: y(active, since 108s), standbys: x 2024-09-10T18:55:53.240 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-10T18:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: laundry: 2 daemons active (1 hosts) 2024-09-10T18:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: data: 2024-09-10T18:55:53.241 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pools: 13 pools, 356 pgs 2024-09-10T18:55:53.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:55:53.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:53.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pgs: 62.921% pgs unknown 2024-09-10T18:55:53.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 224 unknown 2024-09-10T18:55:53.242 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 132 active+clean 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: io: 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: client: 639 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.243 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: cluster: 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: id: d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 17 pool(s) do not have an application enabled 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: services: 2024-09-10T18:55:53.244 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 5m) 2024-09-10T18:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mgr: y(active, since 110s), standbys: x 2024-09-10T18:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-10T18:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: foo: 6 portals active (1 hosts, 3 zones) 2024-09-10T18:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-10T18:55:53.245 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: data: 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pools: 30 pools, 900 pgs 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pgs: 44.889% pgs unknown 2024-09-10T18:55:53.246 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 39.667% pgs not active 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 404 unknown 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 357 creating+peering 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 139 active+clean 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: io: 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: client: 933 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-10T18:55:53.247 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: cluster: 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: id: d7c9fa7c-6fa4-11ef-bcea-c7b262605968 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: health: HEALTH_WARN 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 5 pool(s) do not have an application enabled 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:55:53.248 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: services: 2024-09-10T18:55:53.249 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mon: 3 daemons, quorum a,b,c (age 6m) 2024-09-10T18:55:53.249 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: mgr: y(active, since 112s), standbys: x 2024-09-10T18:55:53.249 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: osd: 8 osds: 8 up (since 2m), 8 in (since 2m) 2024-09-10T18:55:53.249 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: foo: 16 portals active (1 hosts, 3 zones) 2024-09-10T18:55:53.249 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: laundry: 1 daemon active (1 hosts) 2024-09-10T18:55:53.667 INFO:tasks.workunit.client.0.smithi107.stdout: really_mean_it": true}]: dispatch 2024-09-10T18:55:53.667 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:51.664500+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.068654+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.068937+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.069065+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.069249+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.069358+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.069434+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.074127+0000 mon.c [INF] from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.082474+0000 mon.c [INF] from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.088742+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.668 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.089375+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.118040+0000 mon.b [INF] from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.129331+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.132525+0000 mon.c [INF] from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.156936+0000 mon.b [INF] from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.186697+0000 mon.c [INF] from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.187084+0000 client.admin [INF] onexx 2024-09-10T18:55:53.669 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.416205+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.416809+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.417256+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.417681+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.418103+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.418517+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.418931+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.419319+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.419751+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.420172+0000 mon.c [INF] from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.442803+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.443715+0000 mon.c [INF] from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.670 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.444326+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.444462+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.445262+0000 mon.b [INF] from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.671 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.445448+0000 mon.a [INF] from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:55:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: onexx 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.929 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: onexx 2024-09-10T18:55:53.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyECPP_smithi107-80569-1", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:53.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWrite_smithi107-80053-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosIoEC_smithi107-80087-16", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: osdmap e70: 8 total, 8 up, 8 in 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:53.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: onexx 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.975 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:55:53.976 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["onexx"]}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:53.977 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.1 deep-scrub starts 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.9 deep-scrub starts 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.1 deep-scrub ok 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.9 deep-scrub ok 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.2 deep-scrub starts 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.2 deep-scrub ok 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: pgmap v68: 588 pgs: 128 creating+peering, 40 unknown, 420 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2154875551' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.6 deep-scrub starts 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.6 deep-scrub ok 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.7 deep-scrub starts 2024-09-10T18:55:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: 17.7 deep-scrub ok 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:54.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: twoxx 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.1 deep-scrub starts 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.9 deep-scrub starts 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.1 deep-scrub ok 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.9 deep-scrub ok 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.2 deep-scrub starts 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.2 deep-scrub ok 2024-09-10T18:55:54.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: pgmap v68: 588 pgs: 128 creating+peering, 40 unknown, 420 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2154875551' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.6 deep-scrub starts 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.6 deep-scrub ok 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.7 deep-scrub starts 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: 17.7 deep-scrub ok 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: twoxx 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.971 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1197013557' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:54.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1197013557' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:54.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:54.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:54.975 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.1 deep-scrub starts 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.9 deep-scrub starts 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.1 deep-scrub ok 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.9 deep-scrub ok 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.2 deep-scrub starts 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.2 deep-scrub ok 2024-09-10T18:55:55.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: pgmap v68: 588 pgs: 128 creating+peering, 40 unknown, 420 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 MiB/s rd, 23 MiB/s wr, 515 op/s 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2154875551' entity='client.admin' cmd=[{"prefix":"status"}]: dispatch 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.6 deep-scrub starts 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.6 deep-scrub ok 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.7 deep-scrub starts 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: 17.7 deep-scrub ok 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: osdmap e71: 8 total, 8 up, 8 in 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.176 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["onexx"]}]': finished 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: twoxx 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["twoxx"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:55.177 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoECPP_smithi107-80108-22", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosIoEC_smithi107-80087-16", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4048747428' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-2","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolQuotaPP_smithi107-80070-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosStatEC_smithi107-80486-6", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: osdmap e72: 8 total, 8 up, 8 in 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1197013557' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1939435098' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:55.178 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]: dispatch 2024-09-10T18:55:55.179 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.683336+0000 mon.a [INF] from='client.? 172.21.15.107:0/1214483473' entit api_io_pp: Running main() from gmock_main.cc 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [==========] Running 37 tests from 2 test suites. 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] Global test environment set-up. 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: seed 80108 2024-09-10T18:55:55.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TooBigPP 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.TooBigPP (0 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SimpleWritePP 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.SimpleWritePP (169 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadOpPP 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadOpPP (7 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.SparseReadOpPP 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.SparseReadOpPP (2 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP (5 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RoundTripPP2 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RoundTripPP2 (2 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.Checksum 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.Checksum (2 ms) 2024-09-10T18:55:55.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.ReadIntoBufferlist 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.ReadIntoBufferlist (5 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.OverlappingWriteRoundTripPP 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.OverlappingWriteRoundTripPP (6 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP (3 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.WriteFullRoundTripPP2 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.WriteFullRoundTripPP2 (4 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.AppendRoundTripPP 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.AppendRoundTripPP (5 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.TruncTestPP 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.TruncTestPP (3 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RemoveTestPP 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RemoveTestPP (3 ms) 2024-09-10T18:55:55.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrsRoundTripPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrsRoundTripPP (2 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.RmXattrPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.RmXattrPP (6 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.XattrListPP (3 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtPP (5 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtDNEPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtDNEPP (4 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoPP.CmpExtMismatchPP 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoPP.CmpExtMismatchPP (3 ms) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 20 tests from LibRadosIoPP (240 ms total) 2024-09-10T18:55:55.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SimpleWritePP 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SimpleWritePP (1079 ms) 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.ReadOpPP 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.ReadOpPP (11 ms) 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.SparseReadOpPP 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.SparseReadOpPP (5 ms) 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP (3 ms) 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RoundTripPP2 2024-09-10T18:55:55.289 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RoundTripPP2 (5 ms) 2024-09-10T18:55:55.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] L open_pools_parallel: process_1_[80941]: starting. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: creating pool ceph_test_rados_open_pools_parallel.smithi107-80897 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: created object 0... 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: created object 25... 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: created object 49... 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: finishing. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_1_[80941]: shutting down. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[80942]: starting. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[80942]: rados_pool_create. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[80942]: rados_ioctx_create. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_2_[80942]: shutting down. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: starting. 2024-09-10T18:55:55.451 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: creating pool ceph_test_rados_open_pools_parallel.smithi107-80897 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: created object 0... 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: created object 25... 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: created object 49... 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: finishing. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_3_[81911]: shutting down. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[81912]: starting. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[81912]: rados_pool_create. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[81912]: rados_ioctx_create. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: process_4_[81912]: shutting down. 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******************************* 2024-09-10T18:55:55.452 INFO:tasks.workunit.client.0.smithi107.stdout: open_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:55:55.689 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: starting. 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: creating pool ceph_test_rados_delete_pools_parallel.smithi107-80930 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: created object 0... 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: created object 25... 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: created object 49... 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: finishing. 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_1_[80951]: shutting down. 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[80953]: starting. 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[80953]: deleting pool ceph_test_rados_delete_pools_parallel.smithi107-80930 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_2_[80953]: shutting down. 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:55:55.690 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: starting. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: creating pool ceph_test_rados_delete_pools_parallel.smithi107-80930 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: created object 0... 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: created object 25... 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: created object 49... 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: finishing. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_3_[81908]: shutting down. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: starting. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: listing objects. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: listed object 0... 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: listed object 25... 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: saw 50 objects 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_5_[81910]: shutting down. 2024-09-10T18:55:55.691 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[81909]: starting. 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[81909]: deleting pool ceph_test_rados_delete_pools_parallel.smithi107-80930 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: process_4_[81909]: shutting down. 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******************************* 2024-09-10T18:55:55.692 INFO:tasks.workunit.client.0.smithi107.stdout: delete_pools_parallel: ******* SUCCESS ********** 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.3 deep-scrub starts 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.3 deep-scrub ok 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.5 deep-scrub starts 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.5 deep-scrub ok 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.4 deep-scrub starts 2024-09-10T18:55:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: 17.4 deep-scrub ok 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: pgmap v71: 660 pgs: 64 creating+peering, 240 unknown, 356 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 24 MiB/s wr, 545 op/s 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/711679657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.3 deep-scrub starts 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.3 deep-scrub ok 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.5 deep-scrub starts 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.5 deep-scrub ok 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.4 deep-scrub starts 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: 17.4 deep-scrub ok 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: pgmap v71: 660 pgs: 64 creating+peering, 240 unknown, 356 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 24 MiB/s wr, 545 op/s 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/711679657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.3 deep-scrub starts 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.3 deep-scrub ok 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.5 deep-scrub starts 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.5 deep-scrub ok 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.4 deep-scrub starts 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: 17.4 deep-scrub ok 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: pgmap v71: 660 pgs: 64 creating+peering, 240 unknown, 356 active+clean; 97 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 MiB/s rd, 24 MiB/s wr, 545 op/s 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:55:56.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: osdmap e73: 8 total, 8 up, 8 in 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/711679657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.711 INFO:tasks.workunit.client.0.smithi107.stdout:y='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:52.917802+0000 mon.a [INF] from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:54.705761+0000 mon.a [INF] from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:54.706163+0000 mon.a [INF] from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:54.710644+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["twoxx"]}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.177289+0000 mon.a [INF] from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockECPP_smithi107-80149-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.177738+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-4-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.182756+0000 mon.c [INF] from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.446567+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.446655+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.447337+0000 mon.b [INF] from='client.? 172.21.15.107:0/1973597922' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.683563+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "PoolQuotaPP_smithi107-80070-3", "field": "max_bytes", "val": "4096"}]': finished 2024-09-10T18:55:56.712 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.683676+0000 mon.a [INF] from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosLockEC_smithi107-80150-10", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:55:56.713 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.684640+0000 mon.b [INF] from='client.? 172.21.15.107:0/711679657' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:56.713 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.685142+0000 mon.a [INF] from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.687010+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.693599+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]: dispatch 2024-09-10T18:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:55.693781+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.175 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.180703+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.180794+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.435012+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.435336+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.435428+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.435677+0000 mon.c [INF] from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.436714+0000 mon.b [INF] from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.437126+0000 mon.c [INF] from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.437398+0000 mon.c [INF] from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.438304+0000 mon.c [INF] from='client.? 172.21.15.107:0/3717627209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.440253+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.176 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.440581+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: 17.8 deep-scrub starts 2024-09-10T18:55:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: 17.8 deep-scrub ok 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3717627209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: threexx 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: 17.8 deep-scrub starts 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: 17.8 deep-scrub ok 2024-09-10T18:55:57.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3717627209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: threexx 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: 17.0 deep-scrub starts 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: 17.0 deep-scrub ok 2024-09-10T18:55:57.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:57.470 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: 17.0 deep-scrub starts 2024-09-10T18:55:57.470 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: 17.0 deep-scrub ok 2024-09-10T18:55:57.470 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: 17.8 deep-scrub starts 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: 17.8 deep-scrub ok 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatECPP_smithi107-80566-7", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosStatEC_smithi107-80486-6", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyECPP_smithi107-80569-1"}]': finished 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafe_smithi107-80053-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: osdmap e74: 8 total, 8 up, 8 in 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3717627209' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: threexx 2024-09-10T18:55:57.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: 17.0 deep-scrub starts 2024-09-10T18:55:57.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: 17.0 deep-scrub ok 2024-09-10T18:55:57.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:55:58.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.440901+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_iibRadosIoECPP.OverlappingWriteRoundTripPP 2024-09-10T18:55:58.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.OverlappingWriteRoundTripPP (2 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP (6 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.WriteFullRoundTripPP2 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.WriteFullRoundTripPP2 (5 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.AppendRoundTripPP 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.AppendRoundTripPP (5 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.TruncTestPP 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.TruncTestPP (4 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RemoveTestPP 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RemoveTestPP (3 ms) 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrsRoundTripPP 2024-09-10T18:55:58.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrsRoundTripPP (3 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.RmXattrPP 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.RmXattrPP (7 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.XattrListPP 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.XattrListPP (4 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtPP 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtPP (4 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtDNEPP 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtDNEPP (3 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ RUN ] LibRadosIoECPP.CmpExtMismatchPP 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ OK ] LibRadosIoECPP.CmpExtMismatchPP (4 ms) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] 17 tests from LibRadosIoECPP (1154 ms total) 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: 2024-09-10T18:55:58.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [----------] Global test environment tear-down 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [==========] 37 tests from 2 test suites ran. (15208 ms total) 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io_pp: [ PASSED ] 37 tests. 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: Running main() from gmock_main.cc 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [==========] Running 24 tests from 2 test suites. 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] Global test environment set-up. 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 14 tests from LibRadosIo 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.SimpleWrite 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.SimpleWrite (168 ms) 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.TooBig 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.TooBig (0 ms) 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.ReadTimeout 2024-09-10T18:55:58.205 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: no timeout :/ 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.ReadTimeout (189 ms) 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RoundTrip 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RoundTrip (9 ms) 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.Checksum 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.Checksum (5 ms) 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.OverlappingWriteRoundTrip 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.OverlappingWriteRoundTrip (5 ms) 2024-09-10T18:55:58.206 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.WriteFullRoundTrip 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.WriteFullRoundTrip (3 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.AppendRoundTrip 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.AppendRoundTrip (5 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.ZeroLenZero 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.ZeroLenZero (1 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.TruncTest 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.TruncTest (4 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RemoveTest 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RemoveTest (2 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.XattrsRoundTrip 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.XattrsRoundTrip (2 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.RmXattr 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.RmXattr (7 ms) 2024-09-10T18:55:58.207 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIo.XattrIter 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIo.XattrIter (2 ms) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 14 tests from LibRadosIo (403 ms total) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 10 tests from LibRadosIoEC 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.SimpleWrite 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.SimpleWrite (1084 ms) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RoundTrip 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RoundTrip (5 ms) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.OverlappingWriteRoundTrip 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.OverlappingWriteRoundTrip (5 ms) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.WriteFullRoundTrip 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.WriteFullRoundTrip (8 ms) 2024-09-10T18:55:58.208 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.AppendRoundTrip 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.AppendRoundTrip (6 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.TruncTest 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.TruncTest (6 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RemoveTest 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RemoveTest (3 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.XattrsRoundTrip 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.XattrsRoundTrip (2 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.RmXattr 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.RmXattr (8 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ RUN ] LibRadosIoEC.XattrIter 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ OK ] LibRadosIoEC.XattrIter (4 ms) 2024-09-10T18:55:58.209 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] 10 tests from LibRadosIoEC (1131 ms total) 2024-09-10T18:55:58.210 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: 2024-09-10T18:55:58.210 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [----------] Global test environment tear-down 2024-09-10T18:55:58.210 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [==========] 24 tests from 2 test suites ran. (15214 ms total) 2024-09-10T18:55:58.210 INFO:tasks.workunit.client.0.smithi107.stdout: api_io: [ PASSED ] 24 tests. 2024-09-10T18:55:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: pgmap v74: 604 pgs: 73 creating+peering, 24 unknown, 507 active+clean; 121 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 MiB/s wr, 3 op/s 2024-09-10T18:55:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: fourxx 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2101549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: pgmap v74: 604 pgs: 73 creating+peering, 24 unknown, 507 active+clean; 121 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 MiB/s wr, 3 op/s 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: fourxx 2024-09-10T18:55:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2101549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:55:58.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:58.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: pgmap v74: 604 pgs: 73 creating+peering, 24 unknown, 507 active+clean; 121 MiB data, 2.7 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 6.7 MiB/s wr, 3 op/s 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["threexx"]}]': finished 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: fourxx 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["fourxx"]}]: dispatch 2024-09-10T18:55:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosLockEC_smithi107-80150-10", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-4", "overlaypool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: osdmap e75: 8 total, 8 up, 8 in 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/65695595' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3350626000' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2101549098' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]: dispatch 2024-09-10T18:55:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd dump"}]: dispatch 2024-09-10T18:55:58.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:58.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:58.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:55:59] "GET /metrics HTTP/1.1" 200 265705 "" "Prometheus/2.43.0" 2024-09-10T18:55:59.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:55:59.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2"}]: dispatch 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2"}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]: dispatch 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: pool 'PoolQuotaPP_smithi107-80070-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:59.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]': finished 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]': finished 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:59.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: pool 'PoolQuotaPP_smithi107-80070-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]': finished 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.471 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.473 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:55:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"log", "logtext":["fourxx"]}]': finished 2024-09-10T18:55:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:55:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoECPP_smithi107-80108-22"}]': finished 2024-09-10T18:55:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosIoEC_smithi107-80087-16"}]': finished 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-4-cache", "mode": "writeback"}]': finished 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: osdmap e76: 8 total, 8 up, 8 in 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2"}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]: dispatch 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]: dispatch 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: Health check update: 6 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: pool 'PoolQuotaPP_smithi107-80070-3' is full (reached quota's max_bytes: 4 KiB) 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3548376893' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip_smithi107-80053-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMisc_smithi107-80332-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pg_num": 4}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-4"}]': finished 2024-09-10T18:55:59.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: osdmap e77: 8 total, 8 up, 8 in 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3810187745' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/315480214' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3374976717' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"dne","key":"key","value":"value"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:55:59.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:55:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]: dispatch 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: Running main() from gmock_main.cc 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] Global test environment set-up. 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStat 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.Stat 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.Stat (239 ms) 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.StatNS 2024-09-10T18:55:59.933 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.StatNS (17 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.ClusterStat 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.ClusterStat (0 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStat.PoolStat 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStat.PoolStat (2 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStat (258 ms total) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStatEC 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.Stat 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.Stat (1070 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.StatNS 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.StatNS (13 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.ClusterStat 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.ClusterStat (0 ms) 2024-09-10T18:55:59.934 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ RUN ] LibRadosStatEC.PoolStat 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ OK ] LibRadosStatEC.PoolStat (2 ms) 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] 4 tests from LibRadosStatEC (1085 ms total) 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [----------] Global test environment tear-down 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [==========] 8 tests from 2 test suites ran. (16814 ms total) 2024-09-10T18:55:59.935 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat: [ PASSED ] 8 tests. 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: Running main() from gmock_main.cc 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [==========] Running 9 tests from 2 test suites. 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] Global test environment set-up. 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: seed 80566 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPP 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPP (252 ms) 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.Stat2Mtime2PP 2024-09-10T18:55:59.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.Stat2Mtime2PP (11 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.ClusterStatPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.ClusterStatPP (0 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.PoolStatPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.PoolStatPP (5 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatPP.StatPPNS 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatPP.StatPPNS (17 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 5 tests from LibRadosStatPP (285 ms total) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPP (1073 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.ClusterStatPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.ClusterStatPP (1 ms) 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.PoolStatPP 2024-09-10T18:55:59.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.PoolStatPP (3 ms) 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ RUN ] LibRadosStatECPP.StatPPNS 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ OK ] LibRadosStatECPP.StatPPNS (6 ms) 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] 4 tests from LibRadosStatECPP (1083 ms total) 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [----------] Global test environment tear-down 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [==========] 9 tests from 2 test suites ran. (16803 ms total) 2024-09-10T18:55:59.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_stat_pp: [ PASSED ] 9 tests. 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout:_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.441234+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.441973+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosIoEC_smithi107-80087-16"}]: dispatch 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.711406+0000 mon.c [INF] from='client.? 172.21.15.107:0/759064904' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.711567+0000 client.admin [INF] threexx 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: got: 2024-09-10T18:55:56.937700+0000 mon.a [INF] from='client.? ' entity='client.admin' cmd=[{"prefix":"log", "logtext":["threexx"]}]: dispatch 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ OK ] LibRadosCmd.WatchLog (8105 ms) 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] 4 tests from LibRadosCmd (17022 ms total) 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: 2024-09-10T18:56:00.182 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [----------] Global test environment tear-down 2024-09-10T18:56:00.183 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [==========] 4 tests from 1 test suite ran. (17022 ms total) 2024-09-10T18:56:00.183 INFO:tasks.workunit.client.0.smithi107.stdout: api_cmd: [ PASSED ] 4 tests. 2024-09-10T18:56:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: pgmap v78: 624 pgs: 32 creating+peering, 108 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.8 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-09-10T18:56:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/720986161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: pgmap v78: 624 pgs: 32 creating+peering, 108 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.8 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-09-10T18:56:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/720986161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: pgmap v78: 624 pgs: 32 creating+peering, 108 unknown, 484 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail; 1.8 KiB/s rd, 14 MiB/s wr, 5 op/s 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatECPP_smithi107-80566-7"}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-4", "tierpool": "test-rados-api-smithi107-80290-4-cache"}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2825854649' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosStatEC_smithi107-80486-6"}]': finished 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: osdmap e78: 8 total, 8 up, 8 in 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/720986161' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]: dispatch 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:00.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: Running main() from gmock_main.cc 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] Global test environment set-up. 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: seed 80149 2024-09-10T18:56:00.946 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusivePP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusivePP (165 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedPP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedPP (8 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockExclusiveDurPP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockExclusiveDurPP (1080 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockSharedDurPP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockSharedDurPP (1019 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.LockMayRenewPP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.LockMayRenewPP (8 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.UnlockPP 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.UnlockPP (4 ms) 2024-09-10T18:56:00.947 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.ListLockersPP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.ListLockersPP (4 ms) 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockPP.BreakLockPP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockPP.BreakLockPP (3 ms) 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockPP (2292 ms total) 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusivePP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusivePP (547 ms) 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedPP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedPP (3 ms) 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockExclusiveDurPP 2024-09-10T18:56:00.948 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockExclusiveDurPP (1019 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockSharedDurPP 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockSharedDurPP (1008 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.LockMayRenewPP 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.LockMayRenewPP (4 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.UnlockPP 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.UnlockPP (3 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.ListLockersPP 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.ListLockersPP (3 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ RUN ] LibRadosLockECPP.BreakLockPP 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ OK ] LibRadosLockECPP.BreakLockPP (2 ms) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] 8 tests from LibRadosLockECPP (2589 ms total) 2024-09-10T18:56:00.949 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: 2024-09-10T18:56:00.950 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [----------] Global test environment tear-down 2024-09-10T18:56:00.950 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [==========] 16 tests from 2 test suites ran. (17934 ms total) 2024-09-10T18:56:00.950 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock_pp: [ PASSED ] 16 tests. 2024-09-10T18:56:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3693254322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:01.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3693254322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]: dispatch 2024-09-10T18:56:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "overlaypool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1214483473' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockECPP_smithi107-80149-10"}]': finished 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:01.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: osdmap e79: 8 total, 8 up, 8 in 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/20825840' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3693254322' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:01.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:01.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:01.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:01.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:02.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: pgmap v81: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]': finished 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3352356713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:02.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: pgmap v81: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]': finished 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3352356713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: pgmap v81: 552 pgs: 32 creating+peering, 132 unknown, 388 active+clean; 145 MiB data, 2.7 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:02.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "mode": "writeback"}]': finished 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3948704763' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip2_smithi107-80053-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMisc_smithi107-80332-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManaged_smithi107-80386-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: osdmap e80: 8 total, 8 up, 8 in 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3352356713' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]: dispatch 2024-09-10T18:56:02.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:02.973 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: Running main() from gmock_main.cc 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] Global test environment set-up. 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLock 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusive 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockExclusive (248 ms) 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockShared 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockShared (18 ms) 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockExclusiveDur 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockExclusiveDur (1024 ms) 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockSharedDur 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockSharedDur (1098 ms) 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.LockMayRenew 2024-09-10T18:56:02.974 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.LockMayRenew (47 ms) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.Unlock 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.Unlock (20 ms) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.ListLockers 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.ListLockers (7 ms) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLock.BreakLock 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLock.BreakLock (4 ms) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLock (2466 ms total) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLockEC 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusive 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusive (1037 ms) 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockShared 2024-09-10T18:56:02.975 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockShared (15 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockExclusiveDur 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockExclusiveDur (1006 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockSharedDur 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockSharedDur (1004 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.LockMayRenew 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.LockMayRenew (4 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.Unlock 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.Unlock (3 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.ListLockers 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.ListLockers (4 ms) 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ RUN ] LibRadosLockEC.BreakLock 2024-09-10T18:56:02.976 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ OK ] LibRadosLockEC.BreakLock (2 ms) 2024-09-10T18:56:02.977 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] 8 tests from LibRadosLockEC (3075 ms total) 2024-09-10T18:56:02.977 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: 2024-09-10T18:56:02.977 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [----------] Global test environment tear-down 2024-09-10T18:56:02.977 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [==========] 16 tests from 2 test suites ran. (19953 ms total) 2024-09-10T18:56:02.977 INFO:tasks.workunit.client.0.smithi107.stdout: api_lock: [ PASSED ] 16 tests. 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/182289827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/182289827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosWatchNotifyEC_smithi107-80536-12", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4080220247' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosLockEC_smithi107-80150-10"}]': finished 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: osdmap e81: 8 total, 8 up, 8 in 2024-09-10T18:56:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/182289827' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:03.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: pgmap v84: 496 pgs: 26 creating+peering, 46 unknown, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/972784032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: pgmap v84: 496 pgs: 26 creating+peering, 46 unknown, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/972784032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:04.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: pgmap v84: 496 pgs: 26 creating+peering, 46 unknown, 424 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: osdmap e82: 8 total, 8 up, 8 in 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/972784032' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.292 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: Running main() from gmock_main.cc 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [==========] Running 16 tests from 2 test suites. 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] Global test environment set-up. 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotify 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotify (1105 ms) 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyECPP.WatchNotifyTimeout 2024-09-10T18:56:05.293 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyECPP.WatchNotifyTimeout (4 ms) 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 2 tests from LibRadosWatchNotifyECPP (1109 ms total) 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/0 (57 ms) 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify/1 (3939 ms) 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 2024-09-10T18:56:05.294 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/0 (7 ms) 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotifyTimeout/1 (6 ms) 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797610176 notify_id 343597383681 notifier_gid 15141 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/0 (8 ms) 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383680 notifier_gid 15141 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2/1 (9 ms) 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383681 notifier_gid 15141 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/0 (10 ms) 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 2024-09-10T18:56:05.295 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383682 notifier_gid 15141 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioWatchNotify2/1 (10 ms) 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383683 notifier_gid 15141 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/0 (16 ms) 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383680 notifier_gid 15141 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.AioNotify/1 (17 ms) 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: trying... 2024-09-10T18:56:05.296 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613584 notify_id 343597383682 notifier_gid 15141 2024-09-10T18:56:05.297 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: timed out 2024-09-10T18:56:05.297 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-10T18:56:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/659046843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3813413898' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: pgmap v87: 488 pgs: 128 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:05.972 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/659046843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3813413898' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: pgmap v87: 488 pgs: 128 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:05.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTrip3_smithi107-80053-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: osdmap e83: 8 total, 8 up, 8 in 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/659046843' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3813413898' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:06.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: pgmap v87: 488 pgs: 128 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 op/s 2024-09-10T18:56:06.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:06.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:06.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.022 INFO:tasks.workunit.client.0.smithi107.stdout: api_w api_watch_notify: Running main() from gmock_main.cc 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [==========] Running 11 tests from 2 test suites. 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] Global test environment set-up. 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify (173 ms) 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch2Delete 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94333013404544 err -107 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch2Delete (6 ms) 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete 2024-09-10T18:56:07.023 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 300 for disconnect notification ... 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94333013424176 err -107 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete (1060 ms) 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776130 cookie 94333013439728 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2 (16 ms) 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchNotify2 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776128 cookie 94333013439728 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchNotify2 (12 ms) 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioNotify 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776131 cookie 94333013480224 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioNotify (8 ms) 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Multi 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776128 cookie 94333013458944 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776128 cookie 94333013460048 2024-09-10T18:56:07.024 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Multi (17 ms) 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.WatchNotify2Timeout 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 292057776128 cookie 94333013458944 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 296352743425 cookie 94333013458944 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.WatchNotify2Timeout (3008 ms) 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.Watch3Timeout 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 1024 for osd to time us out ... 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94333013458944 err -107 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_cb from 24604 notify_id 322122547202 cookie 94333013458944 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.Watch3Timeout (5007 ms) 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotify.AioWatchDelete2 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: waiting up to 30 for disconnect notification ... 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify2_test_errcb cookie 94333013458944 err -107 2024-09-10T18:56:07.025 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotify.AioWatchDelete2 (1004 ms) 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 10 tests from LibRadosWatchNotify (10311 ms total) 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ RUN ] LibRadosWatchNotifyEC.WatchNotify 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: watch_notify_test_cb 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ OK ] LibRadosWatchNotifyEC.WatchNotify (1078 ms) 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] 1 test from LibRadosWatchNotifyEC (1078 ms total) 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [----------] Global test environment tear-down 2024-09-10T18:56:07.026 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [==========] 11 tests from 2 test suites ran. (23900 ms total) 2024-09-10T18:56:07.027 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify: [ PASSED ] 11 tests. 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:56:07.219 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.220 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:07.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosSnapshotsSelfManagedPP_smithi107-80513-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: osdmap e84: 8 total, 8 up, 8 in 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/828034974' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]: dispatch 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:08.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4079366930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2263985490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: pgmap v90: 488 pgs: 47 creating+peering, 2 creating+activating, 49 unknown, 390 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4079366930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2263985490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: pgmap v90: 488 pgs: 47 creating+peering, 2 creating+activating, 49 unknown, 390 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosWatchNotifyEC_smithi107-80536-12"}]': finished 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: osdmap e85: 8 total, 8 up, 8 in 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4079366930' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2263985490' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: pgmap v90: 488 pgs: 47 creating+peering, 2 creating+activating, 49 unknown, 390 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 1 op/s 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:08.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:09.327 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.327 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.327 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:09] "GET /metrics HTTP/1.1" 200 254058 "" "Prometheus/2.43.0" 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3039611728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1904702912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:09.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:09.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:09 smithi162 ceph-mon[33228]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:56:09.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:09.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:56:09.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3039611728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1904702912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[29404]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-4","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppend_smithi107-80053-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: osdmap e86: 8 total, 8 up, 8 in 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3039611728' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1904702912' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80791-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:09.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:09 smithi107 ceph-mon[40355]: osdmap e87: 8 total, 8 up, 8 in 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: pgmap v93: 488 pgs: 64 unknown, 9 creating+activating, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/934687561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2626743838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: pgmap v93: 488 pgs: 64 unknown, 9 creating+activating, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-10T18:56:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/934687561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2626743838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: pgmap v93: 488 pgs: 64 unknown, 9 creating+activating, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.5 KiB/s wr, 7 op/s 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: osdmap e88: 8 total, 8 up, 8 in 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/934687561' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2626743838' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:11.041 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: Running main() from gmock_main.cc 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [==========] Running 8 tests from 2 test suites. 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] Global test environment set-up. 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibradosCWriteOps.NewDelete 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibradosCWriteOps.NewDelete (0 ms) 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 1 test from LibradosCWriteOps (0 ms total) 2024-09-10T18:56:11.042 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.assertExists 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.assertExists (8918 ms) 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteOpAssertVersion 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteOpAssertVersion (3562 ms) 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Xattrs 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Xattrs (2514 ms) 2024-09-10T18:56:11.043 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Write 2024-09-10T18:56:11.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Write (2730 ms) 2024-09-10T18:56:11.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.Exec 2024-09-10T18:56:11.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.Exec (3013 ms) 2024-09-10T18:56:11.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.WriteSame 2024-09-10T18:56:11.044 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.WriteSame (3055 ms) 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ RUN ] LibRadosCWriteOps.CmpExt 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ OK ] LibRadosCWriteOps.CmpExt (4039 ms) 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] 7 tests from LibRadosCWriteOps (27831 ms total) 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [----------] Global test environment tear-down 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [==========] 8 tests from 2 test suites ran. (27832 ms total) 2024-09-10T18:56:11.045 INFO:tasks.workunit.client.0.smithi107.stdout: api_c_write_operations: [ PASSED ] 8 tests. 2024-09-10T18:56:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:11.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:11.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:11.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-7"}]': finished 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTest_smithi107-80053-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleWritePP_smithi107-80070-5","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: osdmap e89: 8 total, 8 up, 8 in 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:11.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:11.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:12.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:13.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: rgw: 1 daemon active (1 hosts, 1 zones) 2024-09-10T18:56:13.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:56:13.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: data: 2024-09-10T18:56:13.302 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pools: 26 pools, 772 pgs 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: objects: 196 objects, 582 KiB 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: usage: 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: pgs: 56.477% pgs unknown 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 26.425% pgs not active 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 436 unknown 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 204 creating+peering 2024-09-10T18:56:13.303 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 132 active+clean 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: io: 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: client: 852 B/s rd, 0 op/s rd, 0 op/s wr 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.StatusFormat (4429 ms) 2024-09-10T18:56:13.304 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ RUN ] LibRadosService.Status 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ OK ] LibRadosService.Status (20034 ms) 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] 4 tests from LibRadosService (30107 ms total) 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [----------] Global test environment tear-down 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [==========] 4 tests from 1 test suite ran. (30107 ms total) 2024-09-10T18:56:13.305 INFO:tasks.workunit.client.0.smithi107.stdout: api_service: [ PASSED ] 4 tests. 2024-09-10T18:56:13.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: pgmap v96: 488 pgs: 96 unknown, 9 creating+activating, 383 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:56:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: pgmap v96: 488 pgs: 96 unknown, 9 creating+activating, 383 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: pgmap v96: 488 pgs: 96 unknown, 9 creating+activating, 383 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: osdmap e90: 8 total, 8 up, 8 in 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: pgmap v98: 360 pgs: 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1726421411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3250843996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:56:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: pgmap v98: 360 pgs: 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1726421411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3250843996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: pgmap v98: 360 pgs: 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.5 KiB/s rd, 1.2 KiB/s wr, 3 op/s 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1726421411' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3250843996' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: osdmap e91: 8 total, 8 up, 8 in 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:14.693 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [==========] Running 4 tests from 1 test suite. 2024-09-10T18:56:14.693 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] Global test environment set-up. 2024-09-10T18:56:14.693 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterEarly 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterEarly (6309 ms) 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.RegisterLate 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.RegisterLate (24 ms) 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Status 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.Status (20026 ms) 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ RUN ] LibRadosServicePP.Close 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: attempt 0 of 20 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ OK ] LibRadosServicePP.Close (5115 ms) 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] 4 tests from LibRadosServicePP (31474 ms total) 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [----------] Global test environment tear-down 2024-09-10T18:56:14.694 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [==========] 4 tests from 1 test suite ran. (31474 ms total) 2024-09-10T18:56:14.695 INFO:tasks.workunit.client.0.smithi107.stdout: api_service_pp: [ PASSED ] 4 tests. 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3946283819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3043903715' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:15.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3946283819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3043903715' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsEC_smithi107-80386-9", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "WaitForSafePP_smithi107-80070-6","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTrip_smithi107-80053-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: osdmap e92: 8 total, 8 up, 8 in 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3946283819' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:15.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:15.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3043903715' entity='client.admin' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]': finished 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:56:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:16.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]': finished 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4"}]': finished 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: osdmap e93: 8 total, 8 up, 8 in 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:16.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: pgmap v101: 496 pgs: 136 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/410057617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/988063033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]': finished 2024-09-10T18:56:17.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:17 smithi162 ceph-mon[33228]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: pgmap v101: 496 pgs: 136 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/410057617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/988063033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]': finished 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[40355]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "service dump"}]: dispatch 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: pgmap v101: 496 pgs: 136 unknown, 360 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-4", "tierpool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6"}]': finished 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: osdmap e94: 8 total, 8 up, 8 in 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/410057617' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.720 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/988063033' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]: dispatch 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2307474006' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "pool2": "LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP_smithi107-80569-6", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP_smithi107-80070-7","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-9", "mode": "writeback"}]': finished 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattr_smithi107-80053-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:17.721 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:17 smithi107 ceph-mon[29404]: osdmap e95: 8 total, 8 up, 8 in 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout:atch_notify_pp: flushed 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/0 (3006 ms) 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: trying... 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613504 notify_id 356482285572 notifier_gid 15141 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: timed out 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushed 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify2Timeout/1 (3010 ms) 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 2024-09-10T18:56:18.235 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: List watches 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613504 notify_id 369367187459 notifier_gid 15141 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: done 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/0 (3017 ms) 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ RUN ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: List watches 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 2024-09-10T18:56:18.236 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: handle_notify cookie 94301797613504 notify_id 382252089348 notifier_gid 15141 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: notify2 done 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: watch_check 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: unwatch2 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: flushing 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: done 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ OK ] LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP.WatchNotify3/1 (3020 ms) 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] 14 tests from LibRadosWatchNotifyPPTests/LibRadosWatchNotifyPP (16133 ms total) 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [----------] Global test environment tear-down 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [==========] 16 tests from 2 test suites ran. (35086 ms total) 2024-09-10T18:56:18.237 INFO:tasks.workunit.client.0.smithi107.stdout: api_watch_notify_pp: [ PASSED ] 16 tests. 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: pgmap v104: 528 pgs: 66 creating+peering, 47 unknown, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:56:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: pgmap v104: 528 pgs: 66 creating+peering, 47 unknown, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:56:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:18.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: pgmap v104: 528 pgs: 66 creating+peering, 47 unknown, 415 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1023 B/s wr, 2 op/s 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: osdmap e96: 8 total, 8 up, 8 in 2024-09-10T18:56:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:19.407 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:19] "GET /metrics HTTP/1.1" 200 254285 "" "Prometheus/2.43.0" 2024-09-10T18:56:19.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:19.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:19.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:19.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:19.408 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[40355]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:56:19.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:19.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:19.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:19.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:19.416 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:19 smithi107 ceph-mon[29404]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:56:19.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]: dispatch 2024-09-10T18:56:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:19 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-9"}]': finished 2024-09-10T18:56:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:19 smithi162 ceph-mon[33228]: osdmap e97: 8 total, 8 up, 8 in 2024-09-10T18:56:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2833575229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3343531588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: pgmap v108: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2833575229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3343531588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: pgmap v108: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:20.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:20.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2833575229' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3343531588' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: pgmap v108: 428 pgs: 64 unknown, 364 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP2_smithi107-80070-8","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrIter_smithi107-80053-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: osdmap e98: 8 total, 8 up, 8 in 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80337-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:21.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: osdmap e99: 8 total, 8 up, 8 in 2024-09-10T18:56:21.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:21.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:21.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.079 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:56:22.079 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] Global test environment set-up. 2024-09-10T18:56:22.079 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscVersion.Version 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscVersion.Version (0 ms) 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectFailure 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: unable to get monitor info from DNS SRV with service name: ceph-mon 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:55:43.122+0000 7f402cac3600 -1 failed for service _ceph-mon._tcp 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:55:43.122+0000 7f402cac3600 -1 monclient: get_monmap_and_config cannot identify monitors to contact 2024-09-10T18:56:22.080 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectFailure (53 ms) 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscConnectFailure.ConnectTimeout 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscConnectFailure.ConnectTimeout (5014 ms) 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 2 tests from LibRadosMiscConnectFailure (5067 ms total) 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscPool 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMiscPool.PoolCreationRace 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 0x55ffcec072d0 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 0x55ffcec0bad0 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: started 2 aios 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: waiting 0x55ffcec072d0 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: waiting 0x55ffcec0bad0 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: done. 2024-09-10T18:56:22.081 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMiscPool.PoolCreationRace (7045 ms) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 1 test from LibRadosMiscPool (7045 ms total) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 8 tests from LibRadosMisc 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.ClusterFSID 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.ClusterFSID (0 ms) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.Exec 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.Exec (51 ms) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.WriteSame 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.WriteSame (9 ms) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.CmpExt 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.CmpExt (3 ms) 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.Applications 2024-09-10T18:56:22.082 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.Applications (4921 ms) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatOSD 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatOSD (0 ms) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.MinCompatClient 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.MinCompatClient (0 ms) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ RUN ] LibRadosMisc.ShutdownRace 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ OK ] LibRadosMisc.ShutdownRace (19453 ms) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] 8 tests from LibRadosMisc (24437 ms total) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [----------] Global test environment tear-down 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [==========] 12 tests from 4 test suites ran. (38998 ms total) 2024-09-10T18:56:22.083 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc: [ PASSED ] 12 tests. 2024-09-10T18:56:22.423 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: pgmap v111: 396 pgs: 64 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3825003284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2165861763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:22.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: pgmap v111: 396 pgs: 64 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3825003284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2165861763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: pgmap v111: 396 pgs: 64 unknown, 332 active+clean; 145 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: osdmap e100: 8 total, 8 up, 8 in 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3825003284' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2165861763' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:22.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: 17.2 deep-scrub starts 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: 17.2 deep-scrub ok 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: pgmap v114: 460 pgs: 14 creating+activating, 60 creating+peering, 47 unknown, 339 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 256 KiB/s wr, 2 op/s 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: 17.1 deep-scrub starts 2024-09-10T18:56:23.927 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:23 smithi162 ceph-mon[33228]: 17.1 deep-scrub ok 2024-09-10T18:56:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: 17.2 deep-scrub starts 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: 17.2 deep-scrub ok 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:56:23.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.0"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.1"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.2"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.3"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.4"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.5"}]: dispatch 2024-09-10T18:56:23.971 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.6"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.7"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.8"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "17.9"}]: dispatch 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: 17.2 deep-scrub starts 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: 17.2 deep-scrub ok 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsComplete_smithi107-80053-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.972 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache", "force_nonempty":""}]': finished 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-9","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: osdmap e101: 8 total, 8 up, 8 in 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: pgmap v114: 460 pgs: 14 creating+activating, 60 creating+peering, 47 unknown, 339 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 256 KiB/s wr, 2 op/s 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:23.973 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: 17.1 deep-scrub starts 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[29404]: 17.1 deep-scrub ok 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: pgmap v114: 460 pgs: 14 creating+activating, 60 creating+peering, 47 unknown, 339 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 1022 B/s rd, 256 KiB/s wr, 2 op/s 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: 17.1 deep-scrub starts 2024-09-10T18:56:23.974 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:23 smithi107 ceph-mon[40355]: 17.1 deep-scrub ok 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: 17.9 deep-scrub starts 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: 17.9 deep-scrub ok 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: 17.9 deep-scrub starts 2024-09-10T18:56:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: 17.9 deep-scrub ok 2024-09-10T18:56:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: 17.9 deep-scrub starts 2024-09-10T18:56:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: 17.9 deep-scrub ok 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80337-10-cache", "mode":"readonly", "yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: osdmap e102: 8 total, 8 up, 8 in 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1596785473' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: 17.6 deep-scrub starts 2024-09-10T18:56:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: 17.6 deep-scrub ok 2024-09-10T18:56:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: 17.3 deep-scrub starts 2024-09-10T18:56:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: 17.3 deep-scrub ok 2024-09-10T18:56:26.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]': finished 2024-09-10T18:56:26.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:26.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:26.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:56:26.428 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4129882050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2446849845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: pgmap v117: 452 pgs: 14 creating+activating, 29 creating+peering, 78 unknown, 331 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:56:26.429 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: 17.6 deep-scrub starts 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: 17.6 deep-scrub ok 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: 17.3 deep-scrub starts 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: 17.3 deep-scrub ok 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]': finished 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4129882050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2446849845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: pgmap v117: 452 pgs: 14 creating+activating, 29 creating+peering, 78 unknown, 331 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: 17.6 deep-scrub starts 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: 17.6 deep-scrub ok 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: 17.3 deep-scrub starts 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: 17.3 deep-scrub ok 2024-09-10T18:56:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80337-10", "tierpool":"test-rados-api-smithi107-80337-10-cache"}]': finished 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: osdmap e103: 8 total, 8 up, 8 in 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1268370679' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4129882050' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2446849845' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: pgmap v117: 452 pgs: 14 creating+activating, 29 creating+peering, 78 unknown, 331 active+clean; 146 MiB data, 2.8 GiB used, 712 GiB / 715 GiB avail; 767 B/s rd, 1 op/s 2024-09-10T18:56:26.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: Running main() from gmock_main.cc 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [==========] Running 9 tests from 1 test suite. 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] Global test environment set-up. 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] 9 tests from LibRadosPools 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolList 2024-09-10T18:56:27.101 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolList (6781 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup (3584 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookup2 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookup2 (3013 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolLookupOtherInstance 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolLookupOtherInstance (2486 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolReverseLookupOtherInstance 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolReverseLookupOtherInstance (3268 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolDelete 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolDelete (4784 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateDelete 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateDelete (5075 ms) 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolCreateWithCrushRule 2024-09-10T18:56:27.102 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolCreateWithCrushRule (6195 ms) 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ RUN ] LibRadosPools.PoolGetBaseTier 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ OK ] LibRadosPools.PoolGetBaseTier (8851 ms) 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] 9 tests from LibRadosPools (44037 ms total) 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [----------] Global test environment tear-down 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [==========] 9 tests from 1 test suite ran. (44037 ms total) 2024-09-10T18:56:27.103 INFO:tasks.workunit.client.0.smithi107.stdout: api_pool: [ PASSED ] 9 tests. 2024-09-10T18:56:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]': finished 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:27.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:27.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:27.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:27.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]': finished 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsEC_smithi107-80386-9"}]': finished 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripSparseReadPP_smithi107-80070-10","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-11", "mode": "writeback"}]': finished 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafe_smithi107-80053-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: osdmap e104: 8 total, 8 up, 8 in 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:27.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: pgmap v119: 420 pgs: 14 creating+activating, 31 unknown, 375 active+clean; 194 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 509 B/s rd, 12 MiB/s wr, 1 op/s 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: 17.7 deep-scrub starts 2024-09-10T18:56:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: 17.7 deep-scrub ok 2024-09-10T18:56:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: pgmap v119: 420 pgs: 14 creating+activating, 31 unknown, 375 active+clean; 194 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 509 B/s rd, 12 MiB/s wr, 1 op/s 2024-09-10T18:56:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:56:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: 17.7 deep-scrub starts 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: 17.7 deep-scrub ok 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: pgmap v119: 420 pgs: 14 creating+activating, 31 unknown, 375 active+clean; 194 MiB data, 2.9 GiB used, 712 GiB / 715 GiB avail; 509 B/s rd, 12 MiB/s wr, 1 op/s 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: osdmap e105: 8 total, 8 up, 8 in 2024-09-10T18:56:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: 17.7 deep-scrub starts 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: 17.7 deep-scrub ok 2024-09-10T18:56:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: 17.8 deep-scrub starts 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: 17.8 deep-scrub ok 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:56:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: 17.4 deep-scrub starts 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: 17.4 deep-scrub ok 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:29 smithi162 ceph-mon[33228]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:56:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:29] "GET /metrics HTTP/1.1" 200 254285 "" "Prometheus/2.43.0" 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: 17.8 deep-scrub starts 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: 17.8 deep-scrub ok 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: 17.4 deep-scrub starts 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: 17.4 deep-scrub ok 2024-09-10T18:56:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[29404]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: 17.8 deep-scrub starts 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: 17.8 deep-scrub ok 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: osdmap e106: 8 total, 8 up, 8 in 2024-09-10T18:56:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]: dispatch 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: 17.4 deep-scrub starts 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: 17.4 deep-scrub ok 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedEC_smithi107-80386-14", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-11"}]': finished 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1905860982' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValue_smithi107-80053-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1912193953' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsCompletePP_smithi107-80070-11","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:29.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:29 smithi107 ceph-mon[40355]: osdmap e107: 8 total, 8 up, 8 in 2024-09-10T18:56:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-mon[33228]: 17.0 deep-scrub starts 2024-09-10T18:56:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-mon[33228]: 17.0 deep-scrub ok 2024-09-10T18:56:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-mon[33228]: pgmap v123: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-10T18:56:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-mon[33228]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:56:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[29404]: 17.0 deep-scrub starts 2024-09-10T18:56:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[29404]: 17.0 deep-scrub ok 2024-09-10T18:56:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[29404]: pgmap v123: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-10T18:56:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[29404]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:56:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[40355]: 17.0 deep-scrub starts 2024-09-10T18:56:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[40355]: 17.0 deep-scrub ok 2024-09-10T18:56:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[40355]: pgmap v123: 396 pgs: 72 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.0 KiB/s rd, 18 MiB/s wr, 8 op/s 2024-09-10T18:56:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:30 smithi107 ceph-mon[40355]: osdmap e108: 8 total, 8 up, 8 in 2024-09-10T18:56:31.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4078527086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2829708814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:31.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:31.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:56:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4078527086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2829708814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: osdmap e109: 8 total, 8 up, 8 in 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4078527086' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2829708814' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:31.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:32.553 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: 17.5 deep-scrub starts 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: pgmap v126: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s wr, 1 op/s 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: 17.5 deep-scrub ok 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:56:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: 17.5 deep-scrub starts 2024-09-10T18:56:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: pgmap v126: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s wr, 1 op/s 2024-09-10T18:56:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: 17.5 deep-scrub ok 2024-09-10T18:56:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: 17.5 deep-scrub starts 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: pgmap v126: 396 pgs: 104 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 6.0 MiB/s wr, 1 op/s 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: 17.5 deep-scrub ok 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: Health check update: 5 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "Flush_smithi107-80053-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "IsSafePP_smithi107-80070-12","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: osdmap e110: 8 total, 8 up, 8 in 2024-09-10T18:56:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:33.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:33 smithi162 ceph-mon[33228]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:56:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[29404]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:56:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[40355]: osdmap e111: 8 total, 8 up, 8 in 2024-09-10T18:56:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[29404]: pgmap v129: 332 pgs: 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s wr, 0 op/s 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[29404]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[40355]: pgmap v129: 332 pgs: 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s wr, 0 op/s 2024-09-10T18:56:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[40355]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:56:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:34 smithi162 ceph-mon[33228]: pgmap v129: 332 pgs: 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 4.5 KiB/s wr, 0 op/s 2024-09-10T18:56:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:34 smithi162 ceph-mon[33228]: osdmap e112: 8 total, 8 up, 8 in 2024-09-10T18:56:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:34.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:36 smithi162 ceph-mon[33228]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:56:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:36 smithi162 ceph-mon[33228]: pgmap v132: 396 pgs: 64 unknown, 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-10T18:56:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:36.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[40355]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[40355]: pgmap v132: 396 pgs: 64 unknown, 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/374688149' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReturnValuePP_smithi107-80070-13","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/893275635' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsync_smithi107-80053-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[29404]: osdmap e113: 8 total, 8 up, 8 in 2024-09-10T18:56:36.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[29404]: pgmap v132: 396 pgs: 64 unknown, 6 creating+activating, 15 creating+peering, 311 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-10T18:56:36.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:37 smithi162 ceph-mon[33228]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:56:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:37.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[29404]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:56:37.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:37.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:37.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[40355]: osdmap e114: 8 total, 8 up, 8 in 2024-09-10T18:56:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: pgmap v134: 332 pgs: 6 creating+activating, 326 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 52 KiB/s wr, 52 op/s 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/506109387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:38.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:38.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: pgmap v134: 332 pgs: 6 creating+activating, 326 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 52 KiB/s wr, 52 op/s 2024-09-10T18:56:38.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/506109387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: pgmap v134: 332 pgs: 6 creating+activating, 326 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 52 KiB/s wr, 52 op/s 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: osdmap e115: 8 total, 8 up, 8 in 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/506109387' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFull_smithi107-80053-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:56:38.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2919628976' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushPP_smithi107-80070-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:38.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: osdmap e116: 8 total, 8 up, 8 in 2024-09-10T18:56:38.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:38.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:38.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]: dispatch 2024-09-10T18:56:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:39] "GET /metrics HTTP/1.1" 200 242462 "" "Prometheus/2.43.0" 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]': finished 2024-09-10T18:56:39.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:39.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:56:39.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:39.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]': finished 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-13", "mode": "writeback"}]': finished 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: osdmap e117: 8 total, 8 up, 8 in 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1122043799' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:39.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]: dispatch 2024-09-10T18:56:40.065 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: Running main() from gmock_main.cc 2024-09-10T18:56:40.065 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [==========] Running 12 tests from 4 test suites. 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] Global test environment set-up. 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapList 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapList (3344 ms) 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapRemove 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapRemove (2013 ms) 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.Rollback 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.Rollback (2016 ms) 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshots.SnapGetName 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshots.SnapGetName (1723 ms) 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshots (9097 ms total) 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-10T18:56:40.066 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Snap 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Snap (3912 ms) 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManaged.Rollback 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManaged.Rollback (3976 ms) 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManaged (7888 ms total) 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapList 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapList (3114 ms) 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapRemove 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapRemove (1830 ms) 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.Rollback 2024-09-10T18:56:40.067 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.Rollback (2021 ms) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsEC.SnapGetName 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsEC.SnapGetName (1977 ms) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 4 tests from LibRadosSnapshotsEC (8943 ms total) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Snap 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Snap (4039 ms) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ RUN ] LibRadosSnapshotsSelfManagedEC.Rollback 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ OK ] LibRadosSnapshotsSelfManagedEC.Rollback (4016 ms) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] 2 tests from LibRadosSnapshotsSelfManagedEC (8055 ms total) 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [----------] Global test environment tear-down 2024-09-10T18:56:40.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [==========] 12 tests from 4 test suites ran. (56983 ms total) 2024-09-10T18:56:40.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots: [ PASSED ] 12 tests. 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: pgmap v138: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 121 KiB/s wr, 125 op/s 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1398270644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2376893509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: pgmap v138: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 121 KiB/s wr, 125 op/s 2024-09-10T18:56:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:56:40.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1398270644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2376893509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: pgmap v138: 324 pgs: 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 121 KiB/s wr, 125 op/s 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:40.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedEC_smithi107-80386-14"}]': finished 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "31"}]': finished 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: osdmap e118: 8 total, 8 up, 8 in 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1398270644' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2376893509' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:40.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.0 scrub starts 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.0 scrub ok 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.7 scrub starts 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.7 scrub ok 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.1 scrub starts 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.1 scrub ok 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.3 scrub starts 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: 97.3 scrub ok 2024-09-10T18:56:41.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:41.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.0 scrub starts 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.0 scrub ok 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.7 scrub starts 2024-09-10T18:56:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.7 scrub ok 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.1 scrub starts 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.1 scrub ok 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.3 scrub starts 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: 97.3 scrub ok 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:41.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.0"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.1"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.2"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.3"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.4"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.5"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.6"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.7"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.8"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "97.9"}]: dispatch 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.0 scrub starts 2024-09-10T18:56:41.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.0 scrub ok 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.7 scrub starts 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.7 scrub ok 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.1 scrub starts 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.1 scrub ok 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.3 scrub starts 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: 97.3 scrub ok 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "FlushAsyncPP_smithi107-80070-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSame_smithi107-80053-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: osdmap e119: 8 total, 8 up, 8 in 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:41.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]: dispatch 2024-09-10T18:56:42.594 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:42 smithi162 ceph-mon[33228]: pgmap v141: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 68 KiB/s wr, 72 op/s 2024-09-10T18:56:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:42 smithi162 ceph-mon[33228]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:56:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[29404]: pgmap v141: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 68 KiB/s wr, 72 op/s 2024-09-10T18:56:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[29404]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:56:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[40355]: pgmap v141: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 0 B/s rd, 68 KiB/s wr, 72 op/s 2024-09-10T18:56:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pg_num","val":"11"}]': finished 2024-09-10T18:56:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[40355]: osdmap e120: 8 total, 8 up, 8 in 2024-09-10T18:56:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/966407205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/966407205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: osdmap e121: 8 total, 8 up, 8 in 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/966407205' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]: dispatch 2024-09-10T18:56:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: 97.8 scrub starts 2024-09-10T18:56:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: 97.8 scrub ok 2024-09-10T18:56:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: pgmap v144: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 4.0 KiB/s wr, 84 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:56:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:56:44.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: 97.8 scrub starts 2024-09-10T18:56:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: 97.8 scrub ok 2024-09-10T18:56:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: pgmap v144: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 4.0 KiB/s wr, 84 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: 97.8 scrub starts 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: 97.8 scrub ok 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: pgmap v144: 388 pgs: 64 unknown, 324 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 57 KiB/s rd, 4.0 KiB/s wr, 84 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/603960715' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStat_smithi107-80053-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP_smithi107-80070-16","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:44.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pg_num_actual", "val": "31"}]': finished 2024-09-10T18:56:44.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: osdmap e122: 8 total, 8 up, 8 in 2024-09-10T18:56:44.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:45 smithi162 ceph-mon[33228]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:56:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:45 smithi162 ceph-mon[33228]: 97.2 scrub starts 2024-09-10T18:56:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:45 smithi162 ceph-mon[33228]: 97.2 scrub ok 2024-09-10T18:56:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[29404]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[29404]: 97.2 scrub starts 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[29404]: 97.2 scrub ok 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[40355]: osdmap e123: 8 total, 8 up, 8 in 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[40355]: 97.2 scrub starts 2024-09-10T18:56:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:45 smithi107 ceph-mon[40355]: 97.2 scrub ok 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: pgmap v147: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: 97.5 scrub starts 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: 97.5 scrub ok 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: 97.9 scrub starts 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: 97.9 scrub ok 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:56:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: pgmap v147: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: 97.5 scrub starts 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: 97.5 scrub ok 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: 97.9 scrub starts 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: 97.9 scrub ok 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: pgmap v147: 324 pgs: 32 unknown, 292 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 56 KiB/s rd, 0 B/s wr, 74 op/s; 31 B/s, 0 objects/s recovering 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: 97.5 scrub starts 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: 97.5 scrub ok 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: 97.9 scrub starts 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: 97.9 scrub ok 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2087458083' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80513-14","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: osdmap e124: 8 total, 8 up, 8 in 2024-09-10T18:56:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:46.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:47 smithi162 ceph-mon[33228]: pgmap v149: 387 pgs: 29 creating+peering, 1 peering, 50 unknown, 307 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:47 smithi162 ceph-mon[33228]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:56:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[29404]: pgmap v149: 387 pgs: 29 creating+peering, 1 peering, 50 unknown, 307 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[29404]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[40355]: pgmap v149: 387 pgs: 29 creating+peering, 1 peering, 50 unknown, 307 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/169247799' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2224813372' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80053-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:47 smithi107 ceph-mon[40355]: osdmap e125: 8 total, 8 up, 8 in 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: 97.4 scrub starts 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: 97.4 scrub ok 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: 97.6 scrub starts 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: 97.6 scrub ok 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:48 smithi162 ceph-mon[33228]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: 97.4 scrub starts 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: 97.4 scrub ok 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: 97.6 scrub starts 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: 97.6 scrub ok 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[29404]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: 97.4 scrub starts 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: 97.4 scrub ok 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: 97.6 scrub starts 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: 97.6 scrub ok 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:48 smithi107 ceph-mon[40355]: osdmap e126: 8 total, 8 up, 8 in 2024-09-10T18:56:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:49] "GET /metrics HTTP/1.1" 200 238562 "" "Prometheus/2.43.0" 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: pgmap v152: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3853534133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: pgmap v152: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3853534133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: pgmap v152: 323 pgs: 1 peering, 322 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: osdmap e127: 8 total, 8 up, 8 in 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3853534133' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:56:50.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:50.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:56:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:50.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatNS_smithi107-80053-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2506689639' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP_smithi107-80070-18","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:50.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:56:50.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: osdmap e128: 8 total, 8 up, 8 in 2024-09-10T18:56:50.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:50.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:56:52.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:52.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:51 smithi162 ceph-mon[33228]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:56:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:52.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:51 smithi107 ceph-mon[29404]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:56:52.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:52.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:51 smithi107 ceph-mon[40355]: osdmap e129: 8 total, 8 up, 8 in 2024-09-10T18:56:52.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: pgmap v155: 355 pgs: 64 unknown, 1 peering, 290 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4285404532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:53.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:56:53.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: pgmap v155: 355 pgs: 64 unknown, 1 peering, 290 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4285404532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: pgmap v155: 355 pgs: 64 unknown, 1 peering, 290 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsECPP_smithi107-80513-15", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: osdmap e130: 8 total, 8 up, 8 in 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4285404532' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:53.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: pgmap v158: 363 pgs: 12 creating+peering, 60 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2"}]: dispatch 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:54.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: pgmap v158: 363 pgs: 12 creating+peering, 60 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:54.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2"}]: dispatch 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: pgmap v158: 363 pgs: 12 creating+peering, 60 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3419493062' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemove_smithi107-80053-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteSamePP2_smithi107-80070-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app1","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:54.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: osdmap e131: 8 total, 8 up, 8 in 2024-09-10T18:56:54.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2"}]: dispatch 2024-09-10T18:56:54.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:54.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:55.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:55.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosMiscPP_smithi107-80302-1","app": "app2","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: osdmap e132: 8 total, 8 up, 8 in 2024-09-10T18:56:55.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"dne","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:55.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: pgmap v161: 299 pgs: 8 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3301958143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1361170338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:56.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: pgmap v161: 299 pgs: 8 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3301958143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1361170338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: pgmap v161: 299 pgs: 8 unknown, 291 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1","value":"value1"}]': finished 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: osdmap e133: 8 total, 8 up, 8 in 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3301958143' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1361170338' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:56.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]: dispatch 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:56:57.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:57.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:57.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:57.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPPNS_smithi107-80070-20","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClass_smithi107-80053-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application set","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key2","value":"value2"}]': finished 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: osdmap e134: 8 total, 8 up, 8 in 2024-09-10T18:56:57.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1298855081' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:57.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]: dispatch 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [==========] Running 31 tests from 7 test suites. 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] Global test environment set-up. 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscVersion.VersionPP 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscVersion.VersionPP (0 ms) 2024-09-10T18:56:58.149 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscVersion (0 ms total) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: seed 80302 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WaitOSDMapPP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WaitOSDMapPP (9 ms) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNamePP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNamePP (264 ms) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongLocatorPP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongLocatorPP (9 ms) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongNSpacePP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongNSpacePP (10 ms) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.LongAttrNamePP 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.LongAttrNamePP (14 ms) 2024-09-10T18:56:58.150 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.ExecPP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.ExecPP (2 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BadFlagsPP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BadFlagsPP (3 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate1PP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate1PP (4 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Operate2PP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Operate2PP (2 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigObjectPP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigObjectPP (8 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AioOperatePP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AioOperatePP (2 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertExistsPP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertExistsPP (5 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.AssertVersionPP 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.AssertVersionPP (8 ms) 2024-09-10T18:56:58.151 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.BigAttrPP 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: osd_max_attr_size = 0 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: osd_max_attr_size == 0; skipping test 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.BigAttrPP (1399 ms) 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyPP 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyPP (364 ms) 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CopyScrubPP 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: waiting for initial deep scrubs... 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: done waiting, doing copies 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: waiting for final deep scrubs... 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: done waiting 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CopyScrubPP (60779 ms) 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.WriteSamePP 2024-09-10T18:56:58.152 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.WriteSamePP (3 ms) 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.CmpExtPP 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.CmpExtPP (4 ms) 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Applications 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Applications (4423 ms) 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatOSD 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatOSD (0 ms) 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.MinCompatClient 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.MinCompatClient (0 ms) 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscPP.Conf 2024-09-10T18:56:58.153 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscPP.Conf (0 ms) 2024-09-10T18:56:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:57 smithi162 ceph-mon[33228]: pgmap v164: 363 pgs: 35 unknown, 328 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:56:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:57 smithi162 ceph-mon[33228]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[29404]: pgmap v164: 363 pgs: 35 unknown, 328 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[29404]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[40355]: pgmap v164: 363 pgs: 35 unknown, 328 active+clean; 217 MiB data, 3.0 GiB used, 712 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool application rm","pool":"LibRadosMiscPP_smithi107-80302-1","app":"app1","key":"key1"}]': finished 2024-09-10T18:56:58.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:57 smithi107 ceph-mon[40355]: osdmap e135: 8 total, 8 up, 8 in 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3162856564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:59.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:59.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:56:59] "GET /metrics HTTP/1.1" 200 238562 "" "Prometheus/2.43.0" 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3162856564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: osdmap e136: 8 total, 8 up, 8 in 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3162856564' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:56:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:56:59.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: pgmap v167: 331 pgs: 64 unknown, 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:00.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:57:00.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:00.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:56:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:00.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:00.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:00.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: pgmap v167: 331 pgs: 64 unknown, 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: pgmap v167: 331 pgs: 64 unknown, 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1669336146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "SimpleStatPP_smithi107-80070-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWrite_smithi107-80053-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80302-24", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: osdmap e137: 8 total, 8 up, 8 in 2024-09-10T18:57:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:56:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:01.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:01.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:00 smithi162 ceph-mon[33228]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:57:01.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:01.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:00 smithi107 ceph-mon[29404]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:57:01.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:01.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:00 smithi107 ceph-mon[40355]: osdmap e138: 8 total, 8 up, 8 in 2024-09-10T18:57:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: pgmap v170: 267 pgs: 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:57:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3299195783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/100155152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:02.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: pgmap v170: 267 pgs: 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3299195783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/100155152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: pgmap v170: 267 pgs: 267 active+clean; 592 KiB data, 3.0 GiB used, 712 GiB / 715 GiB avail 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80302-24", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pool set", "pool": "LibRadosList_smithi107-80132-1", "var": "pgp_num_actual", "val": "30"}]': finished 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "97.b", "id": [0, 7]}]': finished 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: osdmap e139: 8 total, 8 up, 8 in 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3299195783' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/100155152' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:02.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:02 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:03.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:03.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.218 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]: dispatch 2024-09-10T18:57:03.219 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlock_smithi107-80053-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime_smithi107-80070-22","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:03.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix":"osd pool set","pool":"LibRadosList_smithi107-80132-1","var":"pgp_num","val":"11"}]': finished 2024-09-10T18:57:03.220 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: osdmap e140: 8 total, 8 up, 8 in 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1925401861' entity='client.admin' cmd=[{"prefix":"status","format":"json"}]: dispatch 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:03.221 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:02 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 22 tests from LibRadosMiscPP ( api_snapshots_pp: Running main() from gmock_main.cc 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [==========] Running 20 tests from 5 test suites. 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] Global test environment set-up. 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: seed 80513 2024-09-10T18:57:04.163 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapListPP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapListPP (3822 ms) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapRemovePP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapRemovePP (2023 ms) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.RollbackPP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.RollbackPP (1512 ms) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapGetNamePP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapGetNamePP (1725 ms) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsPP.SnapCreateRemovePP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsPP.SnapCreateRemovePP (4029 ms) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 5 tests from LibRadosSnapshotsPP (13111 ms total) 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP 2024-09-10T18:57:04.164 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapPP 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapPP (4127 ms) 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.RollbackPP 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.RollbackPP (4017 ms) 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.SnapOverlapPP (5958 ms) 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.Bug11677 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.Bug11677 (4001 ms) 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.OrderSnap 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.OrderSnap (2055 ms) 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: deleting snap 23 in pool LibRadosSnapshotsSelfManagedPP_smithi107-80513-7 2024-09-10T18:57:04.165 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: waiting for snaps to purge 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedPP.ReusePurgedSnap (17924 ms) 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 6 tests from LibRadosSnapshotsSelfManagedPP (38082 ms total) 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.NotConnected (9 ms) 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosPoolIsInSelfmanagedSnapsMode.FreshInstance (5055 ms) 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 2 tests from LibRadosPoolIsInSelfmanagedSnapsMode (5064 ms total) 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapListPP 2024-09-10T18:57:04.166 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapListPP (3007 ms) 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapRemovePP 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapRemovePP (2004 ms) 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.RollbackPP 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.RollbackPP (2009 ms) 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsECPP.SnapGetNamePP 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsECPP.SnapGetNamePP (1996 ms) 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 4 tests from LibRadosSnapshotsECPP (9016 ms total) 2024-09-10T18:57:04.167 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: pgmap v173: 363 pgs: 32 unknown, 72 creating+peering, 2 peering, 257 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:04 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: pgmap v173: 363 pgs: 32 unknown, 72 creating+peering, 2 peering, 257 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.469 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:04.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: pgmap v173: 363 pgs: 32 unknown, 72 creating+peering, 2 peering, 257 active+clean; 592 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:04.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:04.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:04.469 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: osdmap e141: 8 total, 8 up, 8 in 2024-09-10T18:57:04.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3140610195' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]: dispatch 2024-09-10T18:57:04.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:04.470 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:04.471 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:04.472 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:04 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:57:05.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3270741305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:05.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3270741305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsECPP_smithi107-80513-15"}]': finished 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80302-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWrite_smithi107-80053-26", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosListEC_smithi107-80132-2", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: osdmap e142: 8 total, 8 up, 8 in 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:05.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3270741305' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:05.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: pgmap v176: 300 pgs: 64 unknown, 8 creating+peering, 1 peering, 227 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:06.664 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: pgmap v176: 300 pgs: 64 unknown, 8 creating+peering, 1 peering, 227 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: pgmap v176: 300 pgs: 64 unknown, 8 creating+peering, 1 peering, 227 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OperateMtime2_smithi107-80070-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: osdmap e143: 8 total, 8 up, 8 in 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_ppry=14 expected=14 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:5c6b0b28:::7:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:5c6b0b28:::7:head expected=14:5c6b0b28:::7:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:5c6b0b28:::7:head -> 7 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=7 expected=7 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:c4fdafeb:::6:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:c4fdafeb:::6:head expected=14:c4fdafeb:::6:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:c4fdafeb:::6:head -> 6 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=6 expected=6 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:bd63b0f1:::8:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:bd63b0f1:::8:head expected=14:bd63b0f1:::8:head 2024-09-10T18:57:07.219 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:bd63b0f1:::8:head -> 8 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=8 expected=8 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:b29083e3:::5:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:b29083e3:::5:head expected=14:b29083e3:::5:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:b29083e3:::5:head -> 5 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=5 expected=5 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:89d3ae78:::11:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:89d3ae78:::11:head expected=14:89d3ae78:::11:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:89d3ae78:::11:head -> 11 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=11 expected=11 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:566253c9:::13:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:566253c9:::13:head expected=14:566253c9:::13:head 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:566253c9:::13:head -> 13 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=13 expected=13 2024-09-10T18:57:07.220 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:02547ec2:::1:head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:02547ec2:::1:head expected=14:02547ec2:::1:head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:02547ec2:::1:head -> 1 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=1 expected=1 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : seek to 14:52ea6a34:::10:head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : cursor()=14:52ea6a34:::10:head expected=14:52ea6a34:::10:head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: > 14:52ea6a34:::10:head -> 10 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: : entry=10 expected=10 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.ListObjectsCursor (173 ms) 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjects 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.EnumerateObjects (50649 ms) 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosList.EnumerateObjectsSplit 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 0/5 -> MIN 14:33333333::::head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 1/5 -> 14:33333333::::head 14:66666666::::head 2024-09-10T18:57:07.221 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 2/5 -> 14:66666666::::head 14:99999999::::head 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 3/5 -> 14:99999999::::head 14:cccccccc::::head 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: split 4/5 -> 14:cccccccc::::head MAX 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosList.EnumerateObjectsSplit (21099 ms) 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 7 tests from LibRadosList (72391 ms total) 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 3 tests from LibRadosListEC 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.ListObjects 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjects (1019 ms) 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.ListObjectsNS 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo2,foo3 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo2 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo3 2024-09-10T18:57:07.222 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo1,foo4,foo5 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo4 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo5 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo1 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset foo6,foo7 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo7 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: foo6 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: myset :foo1,:foo2,:foo3,ns1:foo1,ns1:foo4,ns1:foo5,ns2:foo6,ns2:foo7 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo4 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo5 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo7 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns2:foo6 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: ns1:foo1 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo1 2024-09-10T18:57:07.223 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo2 2024-09-10T18:57:07.224 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: :foo3 2024-09-10T18:57:07.224 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsNS (27 ms) 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: pgmap v179: 244 pgs: 12 creating+peering, 4 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:07.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:57:07.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3057856374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:07.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: pgmap v179: 244 pgs: 12 creating+peering, 4 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3057856374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWrite_smithi107-80053-26", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosListEC_smithi107-80132-2", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: osdmap e144: 8 total, 8 up, 8 in 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: pgmap v179: 244 pgs: 12 creating+peering, 4 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: osdmap e145: 8 total, 8 up, 8 in 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/748463749' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]: dispatch 2024-09-10T18:57:07.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3057856374' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:07.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:09] "GET /metrics HTTP/1.1" 200 233217 "" "Prometheus/2.43.0" 2024-09-10T18:57:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80302-24"}]': finished 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "StatRemovePP_smithi107-80070-24","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: osdmap e146: 8 total, 8 up, 8 in 2024-09-10T18:57:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:09.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:09.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListEC.Lis api_aio: Running main() from gmock_main.cc 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [==========] Running 41 tests from 2 test suites. 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] Global test environment set-up. 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 25 tests from LibRadosAio 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.TooBig 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.TooBig (6849 ms) 2024-09-10T18:57:10.186 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleWrite 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleWrite (4621 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.WaitForSafe 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.WaitForSafe (2753 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip (2731 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip2 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip2 (3032 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTrip3 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTrip3 (3009 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripAppend 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripAppend (3013 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RemoveTest 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RemoveTest (3044 ms) 2024-09-10T18:57:10.187 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.XattrsRoundTrip 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.XattrsRoundTrip (3092 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RmXattr 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RmXattr (3112 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.XattrIter 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.XattrIter (2857 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.IsComplete 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.IsComplete (2978 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.IsSafe 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.IsSafe (3033 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.ReturnValue 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.ReturnValue (2957 ms) 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.Flush 2024-09-10T18:57:10.188 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.Flush (3013 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.FlushAsync 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.FlushAsync (3024 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteFull 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteFull (2960 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.RoundTripWriteSame 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.RoundTripWriteSame (3021 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStat 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleStat (3017 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.OperateMtime (3012 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.SimpleStatNS 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.SimpleStatNS (3257 ms) 2024-09-10T18:57:10.189 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.StatRemove 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.StatRemove (2767 ms) 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.ExecuteClass 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.ExecuteClass (3006 ms) 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.MultiWrite 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.MultiWrite (3249 ms) 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAio.AioUnlock 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAio.AioUnlock (2772 ms) 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 25 tests from LibRadosAio (80179 ms total) 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 16 tests from LibRadosAioEC 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleWrite 2024-09-10T18:57:10.190 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleWrite (7029 ms) 2024-09-10T18:57:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: pgmap v182: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: pgmap v182: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:57:10.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:10.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: pgmap v182: 268 pgs: 40 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: osdmap e147: 8 total, 8 up, 8 in 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3009123970' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]: dispatch 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]: dispatch 2024-09-10T18:57:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:11.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4085216488' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4085216488' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:11.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:11.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:57:11.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1395934448' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWrite_smithi107-80053-26"}]': finished 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosListEC_smithi107-80132-2"}]': finished 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1488970906' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/0_smithi107-80302-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: osdmap e148: 8 total, 8 up, 8 in 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4085216488' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:11.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ExecuteClassPP_smithi107-80070-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:11.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForComplete_smithi107-80053-27", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:11.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: osdmap e149: 8 total, 8 up, 8 in 2024-09-10T18:57:11.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:11.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:12.580 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:12 smithi162 ceph-mon[33228]: pgmap v185: 300 pgs: 72 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:12.580 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:12 smithi107 ceph-mon[29404]: pgmap v185: 300 pgs: 72 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:12.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:12 smithi107 ceph-mon[40355]: pgmap v185: 300 pgs: 72 unknown, 228 active+clean; 590 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:13.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.WaitForCompletObjectsStart 2024-09-10T18:57:13.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 1 0 2024-09-10T18:57:13.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 10 0 2024-09-10T18:57:13.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 13 0 2024-09-10T18:57:13.202 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 7 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 14 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 0 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 15 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 11 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 5 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 8 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 6 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 3 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 4 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 12 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 9 0 2024-09-10T18:57:13.203 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2 0 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: have 1 expect one of 0,1,10,11,12,13,14,15,2,3,4,5,6,7,8,9 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListEC.ListObjectsStart (39 ms) 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 3 tests from LibRadosListEC (1085 ms total) 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 1 test from LibRadosListNP 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ RUN ] LibRadosListNP.ListObjectsError 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ OK ] LibRadosListNP.ListObjectsError (3016 ms) 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] 1 test from LibRadosListNP (3016 ms total) 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [----------] Global test environment tear-down 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [==========] 11 tests from 3 test suites ran. (90180 ms total) 2024-09-10T18:57:13.204 INFO:tasks.workunit.client.0.smithi107.stdout: api_list: [ PASSED ] 11 tests. 2024-09-10T18:57:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/831252139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1953906458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/831252139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:57:13.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:57:13.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1953906458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:13.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80132-3","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: osdmap e150: 8 total, 8 up, 8 in 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/831252139' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd=[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]: dispatch 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForComplete_smithi107-80053-27", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4035396105' entity='client.admin' cmd='[{"prefix":"osd pool rm","pool": "test-rados-api-smithi107-80132-3","pool2":"test-rados-api-smithi107-80132-3","yes_i_really_really_mean_it_not_faking": true}]': finished 2024-09-10T18:57:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/1_smithi107-80302-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:13.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: osdmap e151: 8 total, 8 up, 8 in 2024-09-10T18:57:13.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1953906458' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:13.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:13.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: pgmap v188: 300 pgs: 32 creating+peering, 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-09-10T18:57:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: pgmap v188: 300 pgs: 32 creating+peering, 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: pgmap v188: 300 pgs: 32 creating+peering, 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.2 KiB/s rd, 4.0 KiB/s wr, 2 op/s 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "OmapPP_smithi107-80070-26","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: osdmap e152: 8 total, 8 up, 8 in 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]: dispatch 2024-09-10T18:57:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[29404]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[40355]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:15 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-13"}]': finished 2024-09-10T18:57:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:15 smithi162 ceph-mon[33228]: osdmap e153: 8 total, 8 up, 8 in 2024-09-10T18:57:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: pgmap v192: 268 pgs: 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-09-10T18:57:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1976744194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: pgmap v192: 268 pgs: 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1976744194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:16.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: pgmap v192: 268 pgs: 32 unknown, 236 active+clean; 587 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 3.3 KiB/s rd, 4.1 KiB/s wr, 2 op/s 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3501880277' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "LibRadosChecksum/2_smithi107-80302-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: osdmap e154: 8 total, 8 up, 8 in 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]: dispatch 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1976744194' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:16.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: pgmap v195: 268 pgs: 17 creating+peering, 47 unknown, 204 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:18.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: pgmap v195: 268 pgs: 17 creating+peering, 47 unknown, 204 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3676148602' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForComplete_smithi107-80053-27"}]': finished 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiWritePP_smithi107-80070-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: osdmap e155: 8 total, 8 up, 8 in 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: pgmap v195: 268 pgs: 17 creating+peering, 47 unknown, 204 active+clean; 584 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:18.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:19] "GET /metrics HTTP/1.1" 200 233243 "" "Prometheus/2.43.0" 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:19 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:57:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-15","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip_smithi107-80053-28", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: osdmap e156: 8 total, 8 up, 8 in 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:19.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:19.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:19 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: pgmap v198: 268 pgs: 14 creating+peering, 50 unknown, 204 active+clean; 615 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:57:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:57:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:57:20.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.824 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:20.824 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.824 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: pgmap v198: 268 pgs: 14 creating+peering, 50 unknown, 204 active+clean; 615 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:57:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:57:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: pgmap v198: 268 pgs: 14 creating+peering, 50 unknown, 204 active+clean; 615 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 10 KiB/s wr, 1 op/s 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: osdmap e157: 8 total, 8 up, 8 in 2024-09-10T18:57:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosMiscECPP_smithi107-80302-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip_smithi107-80053-28", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3470161351' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "AioUnlockPP_smithi107-80070-28","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: osdmap e158: 8 total, 8 up, 8 in 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:20.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:22.093 INFO:tasks.workunit.client.0.smithi107.stdout:: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP 2024-09-10T18:57:22.093 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.SnapPP 2024-09-10T18:57:22.093 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.SnapPP (4057 ms) 2024-09-10T18:57:22.093 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.RollbackPP 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.RollbackPP (3833 ms) 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ RUN ] LibRadosSnapshotsSelfManagedECPP.Bug11677 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ OK ] LibRadosSnapshotsSelfManagedECPP.Bug11677 (4252 ms) 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] 3 tests from LibRadosSnapshotsSelfManagedECPP (12142 ms total) 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [----------] Global test environment tear-down 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [==========] 20 tests from 5 test suites ran. (98970 ms total) 2024-09-10T18:57:22.094 INFO:tasks.workunit.client.0.smithi107.stdout: api_snapshots_pp: [ PASSED ] 20 tests. 2024-09-10T18:57:22.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.426 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:22 smithi162 ceph-mon[33228]: pgmap v201: 244 pgs: 14 creating+peering, 34 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:22.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:22.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:22.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[29404]: pgmap v201: 244 pgs: 14 creating+peering, 34 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: osdmap e159: 8 total, 8 up, 8 in 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2220033813' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]: dispatch 2024-09-10T18:57:22.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]: dispatch 2024-09-10T18:57:22.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:22 smithi107 ceph-mon[40355]: pgmap v201: 244 pgs: 14 creating+peering, 34 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]': finished 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3345261860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]': finished 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3345261860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosSnapshotsSelfManagedECPP_smithi107-80513-20"}]': finished 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-15", "mode": "writeback"}]': finished 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: osdmap e160: 8 total, 8 up, 8 in 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3345261860' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: Running main() from gmock_main.cc 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [==========] Running 57 tests from 4 test suites. 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] Global test environment set-up. 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.TooBigPP 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.TooBigPP (6863 ms) 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolQuotaPP 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolQuotaPP (16105 ms) 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleWritePP 2024-09-10T18:57:24.068 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleWritePP (6082 ms) 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.WaitForSafePP 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.WaitForSafePP (3109 ms) 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP (3087 ms) 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP2 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP2 (2854 ms) 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripPP3 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripPP3 (2989 ms) 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripSparseReadPP 2024-09-10T18:57:24.069 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripSparseReadPP (3038 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsCompletePP 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.IsCompletePP (2954 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.IsSafePP 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.IsSafePP (3006 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.ReturnValuePP 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.ReturnValuePP (3028 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushPP 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushPP (2961 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.FlushAsyncPP 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.FlushAsyncPP (3019 ms) 2024-09-10T18:57:24.070 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP (3021 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteFullPP2 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteFullPP2 (3006 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP (3259 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripWriteSamePP2 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripWriteSamePP2 (2774 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPPNS 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPPNS (3000 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.SimpleStatPP 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.SimpleStatPP (3248 ms) 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime 2024-09-10T18:57:24.071 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime (2769 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OperateMtime2 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OperateMtime2 (3014 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.StatRemovePP 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.StatRemovePP (3010 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.ExecuteClassPP 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.ExecuteClassPP (3265 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.OmapPP 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.OmapPP (2618 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiWritePP 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiWritePP (3013 ms) 2024-09-10T18:57:24.072 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.AioUnlockPP 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.AioUnlockPP (3006 ms) 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripAppendPP 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio67312 ms total) 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosTwoPoolsECPP.CopyFrom 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosTwoPoolsECPP.CopyFrom (84 ms) 2024-09-10T18:57:24.073 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosTwoPoolsECPP (84 ms total) 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)0, Checksummer::xxhash32, ceph_le > 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Subset 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Subset (30 ms) 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/0.Chunked 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/0.Chunked (7 ms) 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/0 (37 ms total) 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)1, Checksummer::xxhash64, ceph_le > 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Subset 2024-09-10T18:57:24.074 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Subset (33 ms) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/1.Chunked 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/1.Chunked (7 ms) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/1 (40 ms total) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2, where TypeParam = LibRadosChecksumParams<(rados_checksum_type_t)2, Checksummer::crc32c, ceph_le > 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Subset 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Subset (38 ms) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosChecksum/2.Chunked 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosChecksum/2.Chunked (3 ms) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 2 tests from LibRadosChecksum/2 (41 ms total) 2024-09-10T18:57:24.075 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ RUN ] LibRadosMiscECPP.CompareExtentRange 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ OK ] LibRadosMiscECPP.CompareExtentRange (1031 ms) 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] 1 test from LibRadosMiscECPP (1031 ms total) 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [----------] Global test environment tear-down 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [==========] 31 tests from 7 test suites ran. (101003 ms total) 2024-09-10T18:57:24.076 INFO:tasks.workunit.client.0.smithi107.stdout: api_misc_pp: [ PASSED ] 31 tests. 2024-09-10T18:57:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:57:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: pgmap v204: 260 pgs: 16 creating+peering, 16 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:57:24.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: pgmap v204: 260 pgs: 16 creating+peering, 16 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripAppendPP_smithi107-80070-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: osdmap e161: 8 total, 8 up, 8 in 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1673245314' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/924876237' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]: dispatch 2024-09-10T18:57:24.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]: dispatch 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: pgmap v204: 260 pgs: 16 creating+peering, 16 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosMiscECPP_smithi107-80302-36"}]': finished 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip_smithi107-80053-28"}]': finished 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: osdmap e162: 8 total, 8 up, 8 in 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]: dispatch 2024-09-10T18:57:24.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/550432878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:25.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:25.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/550432878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-15"}]': finished 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTrip2_smithi107-80053-29", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: osdmap e163: 8 total, 8 up, 8 in 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/550432878' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:26 smithi162 ceph-mon[33228]: pgmap v207: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:26 smithi162 ceph-mon[33228]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:57:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[29404]: pgmap v207: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[29404]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:57:26.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[40355]: pgmap v207: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RacingRemovePP_smithi107-80070-30","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:26 smithi107 ceph-mon[40355]: osdmap e164: 8 total, 8 up, 8 in 2024-09-10T18:57:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:28 smithi162 ceph-mon[33228]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:57:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:28 smithi162 ceph-mon[33228]: pgmap v210: 236 pgs: 10 creating+peering, 30 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:28.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:28.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[40355]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:57:28.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[40355]: pgmap v210: 236 pgs: 10 creating+peering, 30 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTrip2_smithi107-80053-29", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[29404]: osdmap e165: 8 total, 8 up, 8 in 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:28 smithi107 ceph-mon[29404]: pgmap v210: 236 pgs: 10 creating+peering, 30 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 0 B/s rd, 0 op/s 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/533949477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:29] "GET /metrics HTTP/1.1" 200 233243 "" "Prometheus/2.43.0" 2024-09-10T18:57:29.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/533949477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-17","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: osdmap e166: 8 total, 8 up, 8 in 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/533949477' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: pgmap v213: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:30.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:57:30.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:30.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: pgmap v213: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP_smithi107-80070-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: osdmap e167: 8 total, 8 up, 8 in 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: pgmap v213: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: osdmap e168: 8 total, 8 up, 8 in 2024-09-10T18:57:30.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]: dispatch 2024-09-10T18:57:30.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:30.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]: dispatch 2024-09-10T18:57:31.098 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]': finished 2024-09-10T18:57:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:57:31.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:31.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1729304612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:31.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]': finished 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1729304612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/520286847' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTrip2_smithi107-80053-29"}]': finished 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-17", "mode": "writeback"}]': finished 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: osdmap e169: 8 total, 8 up, 8 in 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1729304612' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:31.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:32.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: pgmap v216: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: pgmap v216: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: pgmap v216: 260 pgs: 22 creating+peering, 42 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripCmpExtPP2_smithi107-80070-32","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppend_smithi107-80053-30", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: osdmap e170: 8 total, 8 up, 8 in 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]: dispatch 2024-09-10T18:57:32.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:33 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:33 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:33.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:33 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: pgmap v219: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 764 B/s wr, 3 op/s 2024-09-10T18:57:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:57:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:57:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:34.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:34.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: pgmap v219: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 764 B/s wr, 3 op/s 2024-09-10T18:57:34.618 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:57:34.619 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:34.619 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:57:34.619 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:34.619 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-17"}]': finished 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: pgmap v219: 228 pgs: 228 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 764 B/s wr, 3 op/s 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: osdmap e171: 8 total, 8 up, 8 in 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppend_smithi107-80053-30", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: osdmap e172: 8 total, 8 up, 8 in 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:36 smithi162 ceph-mon[33228]: pgmap v222: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:57:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[29404]: pgmap v222: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "PoolEIOFlag_smithi107-80070-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: osdmap e173: 8 total, 8 up, 8 in 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3632480137' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]: dispatch 2024-09-10T18:57:36.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:36 smithi107 ceph-mon[40355]: pgmap v222: 268 pgs: 72 unknown, 196 active+clean; 583 KiB data, 2.3 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:37.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:37.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-19","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "PoolEIOFlag_smithi107-80070-33", "var": "eio", "val": "true"}]': finished 2024-09-10T18:57:37.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: osdmap e174: 8 total, 8 up, 8 in 2024-09-10T18:57:37.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:37.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:37.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:38 smithi162 ceph-mon[33228]: pgmap v225: 228 pgs: 6 unknown, 222 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:57:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:57:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[40355]: pgmap v225: 228 pgs: 6 unknown, 222 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: osdmap e175: 8 total, 8 up, 8 in 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/12406186' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]: dispatch 2024-09-10T18:57:38.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:38 smithi107 ceph-mon[29404]: pgmap v225: 228 pgs: 6 unknown, 222 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:39 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:39] "GET /metrics HTTP/1.1" 200 230794 "" "Prometheus/2.43.0" 2024-09-10T18:57:39.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:39.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppend_smithi107-80053-30"}]': finished 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: osdmap e176: 8 total, 8 up, 8 in 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:39 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]': finished 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:57:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: pgmap v228: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]': finished 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:57:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: pgmap v228: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-19", "mode": "writeback"}]': finished 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3207465679' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "MultiReads_smithi107-80070-34","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsComplete_smithi107-80053-31", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: osdmap e177: 8 total, 8 up, 8 in 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: pgmap v228: 260 pgs: 32 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:57:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:57:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:57:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3889516925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:41.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:41.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3889516925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: osdmap e178: 8 total, 8 up, 8 in 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]: dispatch 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsComplete_smithi107-80053-31", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-19"}]': finished 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: osdmap e179: 8 total, 8 up, 8 in 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3889516925' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:42.575 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:42 smithi162 ceph-mon[33228]: pgmap v231: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:42 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:42.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:42 smithi162 ceph-mon[33228]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:57:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[40355]: pgmap v231: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[40355]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:57:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[29404]: pgmap v231: 268 pgs: 40 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:57:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "ReadIntoBufferlist_smithi107-80070-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:42 smithi107 ceph-mon[29404]: osdmap e180: 8 total, 8 up, 8 in 2024-09-10T18:57:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:43 smithi162 ceph-mon[33228]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:57:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[40355]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:57:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[29404]: osdmap e181: 8 total, 8 up, 8 in 2024-09-10T18:57:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: pgmap v234: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:44.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: pgmap v234: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:44.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: pgmap v234: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-21","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: osdmap e182: 8 total, 8 up, 8 in 2024-09-10T18:57:44.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]: dispatch 2024-09-10T18:57:44.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:44.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:44.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: pgmap v237: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:46.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:46.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:46.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: pgmap v237: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2519075118' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsComplete_smithi107-80053-31"}]': finished 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3177901664' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-36","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: osdmap e183: 8 total, 8 up, 8 in 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: pgmap v237: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:46.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2249295995' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:47.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:47.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2249295995' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafe_smithi107-80053-32", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: osdmap e184: 8 total, 8 up, 8 in 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]: dispatch 2024-09-10T18:57:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2249295995' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:47.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:47.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: pgmap v239: 292 pgs: 17 creating+peering, 37 unknown, 238 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s wr, 0 op/s 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]': finished 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3151821652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: pgmap v239: 292 pgs: 17 creating+peering, 37 unknown, 238 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s wr, 0 op/s 2024-09-10T18:57:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]': finished 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3151821652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: pgmap v239: 292 pgs: 17 creating+peering, 37 unknown, 238 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s wr, 0 op/s 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-21", "mode": "writeback"}]': finished 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: osdmap e185: 8 total, 8 up, 8 in 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafe_smithi107-80053-32", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: osdmap e186: 8 total, 8 up, 8 in 2024-09-10T18:57:48.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3151821652' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:48.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]: dispatch 2024-09-10T18:57:48.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:49.378 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:49] "GET /metrics HTTP/1.1" 200 230820 "" "Prometheus/2.43.0" 2024-09-10T18:57:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:49 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:49 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:49 smithi162 ceph-mon[33228]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:57:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:49.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[40355]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:57:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-21"}]': finished 2024-09-10T18:57:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrsRoundTripPP_smithi107-80070-38","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:49 smithi107 ceph-mon[29404]: osdmap e187: 8 total, 8 up, 8 in 2024-09-10T18:57:50.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:50 smithi162 ceph-mon[33228]: pgmap v242: 332 pgs: 40 unknown, 8 creating+activating, 23 creating+peering, 261 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-09-10T18:57:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:50 smithi162 ceph-mon[33228]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:57:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[29404]: pgmap v242: 332 pgs: 40 unknown, 8 creating+activating, 23 creating+peering, 261 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-09-10T18:57:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[29404]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:57:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[40355]: pgmap v242: 332 pgs: 40 unknown, 8 creating+activating, 23 creating+peering, 261 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 510 B/s wr, 1 op/s 2024-09-10T18:57:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[40355]: osdmap e188: 8 total, 8 up, 8 in 2024-09-10T18:57:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:57:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: pgmap v245: 260 pgs: 8 creating+activating, 23 creating+peering, 229 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:52.425 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:57:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: pgmap v245: 260 pgs: 8 creating+activating, 23 creating+peering, 229 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: pgmap v245: 260 pgs: 8 creating+activating, 23 creating+peering, 229 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: osdmap e189: 8 total, 8 up, 8 in 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]: dispatch 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:53.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:53.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:53.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2145430944' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafe_smithi107-80053-32"}]': finished 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-23","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: osdmap e190: 8 total, 8 up, 8 in 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:53.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: pgmap v248: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/789894208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: pgmap v248: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/789894208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: pgmap v248: 228 pgs: 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValue_smithi107-80053-33", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: osdmap e191: 8 total, 8 up, 8 in 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/789894208' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:54.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:54.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:54.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:57:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:57:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3032405386' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:57:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3032405386' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:57:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: osdmap e192: 8 total, 8 up, 8 in 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValue_smithi107-80053-33", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: osdmap e193: 8 total, 8 up, 8 in 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3032405386' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]: dispatch 2024-09-10T18:57:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[40355]: pgmap v251: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]': finished 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[40355]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[29404]: pgmap v251: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]': finished 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:56 smithi107 ceph-mon[29404]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:57:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:56 smithi162 ceph-mon[33228]: pgmap v251: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T18:57:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:56 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:57:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-23", "mode": "writeback"}]': finished 2024-09-10T18:57:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-40","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:56 smithi162 ceph-mon[33228]: osdmap e194: 8 total, 8 up, 8 in 2024-09-10T18:57:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: pgmap v254: 300 pgs: 28 unknown, 9 creating+peering, 263 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:57:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:57:58.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2745775132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: pgmap v254: 300 pgs: 28 unknown, 9 creating+peering, 263 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2745775132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:58 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: pgmap v254: 300 pgs: 28 unknown, 9 creating+peering, 263 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: osdmap e195: 8 total, 8 up, 8 in 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2745775132' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:57:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:58 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:57:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:57:59] "GET /metrics HTTP/1.1" 200 230820 "" "Prometheus/2.43.0" 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:59.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1454267910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:59.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:57:59.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1454267910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:59.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:57:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: osdmap e196: 8 total, 8 up, 8 in 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]: dispatch 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]: dispatch 2024-09-10T18:57:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:57:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/353194951' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValue_smithi107-80053-33"}]': finished 2024-09-10T18:57:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-23"}]': finished 2024-09-10T18:57:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: osdmap e197: 8 total, 8 up, 8 in 2024-09-10T18:57:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1454267910' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:57:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:57:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: pgmap v257: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: pgmap v257: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:58:00.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: pgmap v257: 324 pgs: 32 unknown, 292 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 3.0 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T18:58:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:00.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:00.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-42","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:00.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-Flush_smithi107-80053-34", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:00.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: osdmap e198: 8 total, 8 up, 8 in 2024-09-10T18:58:00.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:58:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/657172626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/657172626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: osdmap e199: 8 total, 8 up, 8 in 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/657172626' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:02.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-mon[33228]: pgmap v260: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-09-10T18:58:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:02 smithi162 ceph-mon[33228]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:58:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[29404]: pgmap v260: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-09-10T18:58:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[29404]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[40355]: pgmap v260: 324 pgs: 64 unknown, 260 active+clean; 584 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s wr, 0 op/s 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "Flush_smithi107-80053-34", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RmXattrPP_smithi107-80070-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-25","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:02 smithi107 ceph-mon[40355]: osdmap e200: 8 total, 8 up, 8 in 2024-09-10T18:58:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[29404]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[40355]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:03 smithi162 ceph-mon[33228]: osdmap e201: 8 total, 8 up, 8 in 2024-09-10T18:58:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: pgmap v263: 396 pgs: 8 unknown, 64 creating+peering, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: pgmap v263: 396 pgs: 8 unknown, 64 creating+peering, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:58:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: pgmap v263: 396 pgs: 8 unknown, 64 creating+peering, 324 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: osdmap e202: 8 total, 8 up, 8 in 2024-09-10T18:58:04.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:04.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:06 smithi162 ceph-mon[33228]: pgmap v267: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 258 B/s wr, 2 op/s 2024-09-10T18:58:06.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[29404]: pgmap v267: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 258 B/s wr, 2 op/s 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: osdmap e203: 8 total, 8 up, 8 in 2024-09-10T18:58:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]: dispatch 2024-09-10T18:58:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]: dispatch 2024-09-10T18:58:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:06 smithi107 ceph-mon[40355]: pgmap v267: 292 pgs: 32 creating+peering, 260 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 258 B/s wr, 2 op/s 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]': finished 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:07.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:07.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:07.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:07.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]': finished 2024-09-10T18:58:07.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4072827034' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"Flush_smithi107-80053-34"}]': finished 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-25", "mode": "writeback"}]': finished 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: osdmap e204: 8 total, 8 up, 8 in 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:07.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:58:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:08 smithi162 ceph-mon[33228]: pgmap v270: 228 pgs: 11 creating+peering, 217 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 259 B/s rd, 0 op/s 2024-09-10T18:58:08.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:08.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[40355]: pgmap v270: 228 pgs: 11 creating+peering, 217 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 259 B/s rd, 0 op/s 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsync_smithi107-80053-35", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: osdmap e205: 8 total, 8 up, 8 in 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]: dispatch 2024-09-10T18:58:08.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:08 smithi107 ceph-mon[29404]: pgmap v270: 228 pgs: 11 creating+peering, 217 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 259 B/s rd, 0 op/s 2024-09-10T18:58:09.349 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:09] "GET /metrics HTTP/1.1" 200 235700 "" "Prometheus/2.43.0" 2024-09-10T18:58:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:09 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:09 smithi162 ceph-mon[33228]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:58:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:09 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[40355]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:09.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-25"}]': finished 2024-09-10T18:58:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[29404]: osdmap e206: 8 total, 8 up, 8 in 2024-09-10T18:58:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:09.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:09 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:58:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: pgmap v273: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:58:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:58:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: pgmap v273: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:58:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:10.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsync_smithi107-80053-35", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2474543940' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RemoveTestPP_smithi107-80070-44","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: osdmap e207: 8 total, 8 up, 8 in 2024-09-10T18:58:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: pgmap v273: 236 pgs: 40 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: osdmap e208: 8 total, 8 up, 8 in 2024-09-10T18:58:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:10.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:12.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:12 smithi162 ceph-mon[33228]: pgmap v276: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:58:12.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[29404]: pgmap v276: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-27","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: osdmap e209: 8 total, 8 up, 8 in 2024-09-10T18:58:12.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:12.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:12.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:12 smithi107 ceph-mon[40355]: pgmap v276: 260 pgs: 64 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:13.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:13.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2368228915' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "XattrListPP_smithi107-80070-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: osdmap e210: 8 total, 8 up, 8 in 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4052248526' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]: dispatch 2024-09-10T18:58:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: pgmap v279: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:14 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:14.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: pgmap v279: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsync_smithi107-80053-35"}]': finished 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: osdmap e211: 8 total, 8 up, 8 in 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: pgmap v279: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:14.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:14 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-46", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: osdmap e212: 8 total, 8 up, 8 in 2024-09-10T18:58:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]: dispatch 2024-09-10T18:58:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]': finished 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:16.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:58:16.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:16.928 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]': finished 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: pgmap v281: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:58:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-27", "mode": "writeback"}]': finished 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: osdmap e213: 8 total, 8 up, 8 in 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-46", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFull_smithi107-80053-36", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: osdmap e214: 8 total, 8 up, 8 in 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:18 smithi162 ceph-mon[33228]: pgmap v284: 244 pgs: 13 creating+peering, 3 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:58:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:18 smithi162 ceph-mon[33228]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:58:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[40355]: pgmap v284: 244 pgs: 13 creating+peering, 3 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:58:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[40355]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[29404]: pgmap v284: 244 pgs: 13 creating+peering, 3 unknown, 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 509 B/s rd, 254 B/s wr, 1 op/s 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[29404]: osdmap e215: 8 total, 8 up, 8 in 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]: dispatch 2024-09-10T18:58:19.395 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:19] "GET /metrics HTTP/1.1" 200 228417 "" "Prometheus/2.43.0" 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:58:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:19.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-27"}]': finished 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: osdmap e216: 8 total, 8 up, 8 in 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: osdmap e217: 8 total, 8 up, 8 in 2024-09-10T18:58:19.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]: dispatch 2024-09-10T18:58:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-10T18:58:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:20.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:20.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:58:20.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.678 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.678 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: pgmap v287: 228 pgs: 228 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 3 op/s 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3244309641' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]: dispatch 2024-09-10T18:58:20.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/766082917' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-46"}]': finished 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFull_smithi107-80053-36"}]': finished 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: osdmap e218: 8 total, 8 up, 8 in 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:20.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:20.720 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:21.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:22.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: pgmap v290: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:22.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:22.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: pgmap v290: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: pgmap v290: 228 pgs: 32 unknown, 196 active+clean; 583 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-29","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStat_smithi107-80053-37", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleWritePP_smithi107-80070-47", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: osdmap e219: 8 total, 8 up, 8 in 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:22.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:23.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:23.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:23 smithi162 ceph-mon[33228]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:58:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:23 smithi107 ceph-mon[29404]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:58:23.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:23 smithi107 ceph-mon[40355]: osdmap e220: 8 total, 8 up, 8 in 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: pgmap v293: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:24.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:24.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:58:24.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:24.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: pgmap v293: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: pgmap v293: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T18:58:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStat_smithi107-80053-37", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleWritePP_smithi107-80070-47", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: osdmap e221: 8 total, 8 up, 8 in 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: osdmap e222: 8 total, 8 up, 8 in 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:24.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:26.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:26 smithi162 ceph-mon[33228]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:58:26.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:26.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:58:26.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[40355]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: osdmap e223: 8 total, 8 up, 8 in 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]: dispatch 2024-09-10T18:58:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:26.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:26 smithi107 ceph-mon[29404]: pgmap v297: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.0 MiB/s wr, 1 op/s 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]': finished 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:27.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:27.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]': finished 2024-09-10T18:58:27.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:27.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-29", "mode": "writeback"}]': finished 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:27.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: osdmap e224: 8 total, 8 up, 8 in 2024-09-10T18:58:27.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2834035367' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/254626038' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:27.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]: dispatch 2024-09-10T18:58:27.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]: dispatch 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:28 smithi162 ceph-mon[33228]: pgmap v300: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 257 B/s wr, 21 op/s 2024-09-10T18:58:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:28.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[29404]: pgmap v300: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 257 B/s wr, 21 op/s 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleWritePP_smithi107-80070-47"}]': finished 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStat_smithi107-80053-37"}]': finished 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: osdmap e225: 8 total, 8 up, 8 in 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:28.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:28 smithi107 ceph-mon[40355]: pgmap v300: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 11 KiB/s rd, 257 B/s wr, 21 op/s 2024-09-10T18:58:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:58:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:29.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:29.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:29] "GET /metrics HTTP/1.1" 200 228417 "" "Prometheus/2.43.0" 2024-09-10T18:58:29.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatNS_smithi107-80053-38", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-WaitForSafePP_smithi107-80070-48", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: osdmap e226: 8 total, 8 up, 8 in 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:30 smithi162 ceph-mon[33228]: pgmap v303: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 255 B/s wr, 22 op/s 2024-09-10T18:58:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:30 smithi162 ceph-mon[33228]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:58:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[29404]: pgmap v303: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 255 B/s wr, 22 op/s 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[29404]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[40355]: pgmap v303: 228 pgs: 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 12 KiB/s rd, 255 B/s wr, 22 op/s 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[40355]: osdmap e227: 8 total, 8 up, 8 in 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:31.115 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:31.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatNS_smithi107-80053-38", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "WaitForSafePP_smithi107-80070-48", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:31.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:31.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: osdmap e228: 8 total, 8 up, 8 in 2024-09-10T18:58:31.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]: dispatch 2024-09-10T18:58:31.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:32.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:32 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:32 smithi162 ceph-mon[33228]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:58:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:32 smithi162 ceph-mon[33228]: pgmap v306: 244 pgs: 16 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[29404]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:58:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[29404]: pgmap v306: 244 pgs: 16 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-29"}]': finished 2024-09-10T18:58:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[40355]: osdmap e229: 8 total, 8 up, 8 in 2024-09-10T18:58:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:32 smithi107 ceph-mon[40355]: pgmap v306: 244 pgs: 16 unknown, 228 active+clean; 8.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:33.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:58:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: osdmap e230: 8 total, 8 up, 8 in 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:33.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: pgmap v309: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:34.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:34.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:34.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: pgmap v309: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: osdmap e231: 8 total, 8 up, 8 in 2024-09-10T18:58:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1501011880' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:34.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]: dispatch 2024-09-10T18:58:34.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]: dispatch 2024-09-10T18:58:34.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: pgmap v309: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:34.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:35.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/541071374' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatNS_smithi107-80053-38"}]': finished 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"WaitForSafePP_smithi107-80070-48"}]': finished 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-31","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: osdmap e232: 8 total, 8 up, 8 in 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:35.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:35.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:36.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: pgmap v311: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemove_smithi107-80053-39", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP_smithi107-80070-49", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: osdmap e233: 8 total, 8 up, 8 in 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: pgmap v314: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:37.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:37.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]': finished 2024-09-10T18:58:37.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:58:37.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:37.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:58:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: pgmap v314: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]': finished 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: osdmap e234: 8 total, 8 up, 8 in 2024-09-10T18:58:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]: dispatch 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: pgmap v314: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemove_smithi107-80053-39", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP_smithi107-80070-49", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-31", "mode": "writeback"}]': finished 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: osdmap e235: 8 total, 8 up, 8 in 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:37.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:39.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:58:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:39] "GET /metrics HTTP/1.1" 200 233328 "" "Prometheus/2.43.0" 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: osdmap e236: 8 total, 8 up, 8 in 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]: dispatch 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-31"}]': finished 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: osdmap e237: 8 total, 8 up, 8 in 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:39.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:39.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: pgmap v318: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:58:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: pgmap v318: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: pgmap v318: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:40.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: osdmap e238: 8 total, 8 up, 8 in 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/299755016' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1821428170' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:40.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]: dispatch 2024-09-10T18:58:40.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]: dispatch 2024-09-10T18:58:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:42.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:58:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:42 smithi162 ceph-mon[33228]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:42.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:42.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[29404]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemove_smithi107-80053-39"}]': finished 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP_smithi107-80070-49"}]': finished 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: osdmap e239: 8 total, 8 up, 8 in 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:42.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:42 smithi107 ceph-mon[40355]: pgmap v321: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-33","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClass_smithi107-80053-40", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripPP2_smithi107-80070-50", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: osdmap e240: 8 total, 8 up, 8 in 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: pgmap v324: 228 pgs: 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:58:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:44.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: pgmap v324: 228 pgs: 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:58:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: pgmap v324: 228 pgs: 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: osdmap e241: 8 total, 8 up, 8 in 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClass_smithi107-80053-40", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripPP2_smithi107-80070-50", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: osdmap e242: 8 total, 8 up, 8 in 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:44.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]: dispatch 2024-09-10T18:58:45.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:45 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]': finished 2024-09-10T18:58:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:45 smithi162 ceph-mon[33228]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:58:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]': finished 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[40355]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-33", "mode": "writeback"}]': finished 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[29404]: osdmap e243: 8 total, 8 up, 8 in 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:45.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: pgmap v327: 244 pgs: 16 unknown, 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: pgmap v327: 244 pgs: 16 unknown, 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: pgmap v327: 244 pgs: 16 unknown, 14 creating+peering, 214 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: osdmap e244: 8 total, 8 up, 8 in 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:58:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-33"}]': finished 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: osdmap e245: 8 total, 8 up, 8 in 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1865456265' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2666008830' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]: dispatch 2024-09-10T18:58:47.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]: dispatch 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:48.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:48.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:48.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: pgmap v330: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripPP2_smithi107-80070-50"}]': finished 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClass_smithi107-80053-40"}]': finished 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: osdmap e246: 8 total, 8 up, 8 in 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:49] "GET /metrics HTTP/1.1" 200 233277 "" "Prometheus/2.43.0" 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1824523328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1824523328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWrite_smithi107-80053-41", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: osdmap e247: 8 total, 8 up, 8 in 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1824523328' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:49.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: pgmap v333: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:58:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:50.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: pgmap v333: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: pgmap v333: 260 pgs: 64 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripPP3_smithi107-80070-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-35","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: osdmap e248: 8 total, 8 up, 8 in 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:58:52.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:52.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWrite_smithi107-80053-41", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: osdmap e249: 8 total, 8 up, 8 in 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: pgmap v336: 236 pgs: 40 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:52.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:53.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:53 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:58:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: osdmap e250: 8 total, 8 up, 8 in 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]: dispatch 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:58:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:53 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]': finished 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:54.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:58:54.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]': finished 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: pgmap v339: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-35", "mode": "writeback"}]': finished 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: osdmap e251: 8 total, 8 up, 8 in 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripSparseReadPP_smithi107-80070-52", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: osdmap e252: 8 total, 8 up, 8 in 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/203743678' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:54.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]: dispatch 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout:te 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.WaitForComplete (6898 ms) 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip (6986 ms) 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTrip2 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTrip2 (7023 ms) 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripAppend 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripAppend (7004 ms) 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.IsComplete 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.IsComplete (7021 ms) 2024-09-10T18:58:55.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.IsSafe 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.IsSafe (7033 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.ReturnValue 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.ReturnValue (7025 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.Flush 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.Flush (6916 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.FlushAsync 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.FlushAsync (7026 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.RoundTripWriteFull 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.RoundTripWriteFull (7031 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStat 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStat (6951 ms) 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.SimpleStatNS 2024-09-10T18:58:55.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.SimpleStatNS (7040 ms) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.StatRemove 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.StatRemove (6954 ms) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.ExecuteClass 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.ExecuteClass (7022 ms) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ RUN ] LibRadosAioEC.MultiWrite 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ OK ] LibRadosAioEC.MultiWrite (7025 ms) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] 16 tests from LibRadosAioEC (111984 ms total) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [----------] Global test environment tear-down 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [==========] 41 tests from 2 test suites ran. (192163 ms total) 2024-09-10T18:58:55.145 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio: [ PASSED ] 41 tests. 2024-09-10T18:58:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:56 smithi162 ceph-mon[33228]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:58:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:56 smithi162 ceph-mon[33228]: pgmap v342: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:56.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[29404]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:58:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[29404]: pgmap v342: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:56.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWrite_smithi107-80053-41"}]': finished 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[40355]: osdmap e253: 8 total, 8 up, 8 in 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[40355]: pgmap v342: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:56.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:57.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:58:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: osdmap e254: 8 total, 8 up, 8 in 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:57.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 764 B/s wr, 2 op/s 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 764 B/s wr, 2 op/s 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: pgmap v344: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 764 B/s wr, 2 op/s 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:58:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: osdmap e255: 8 total, 8 up, 8 in 2024-09-10T18:58:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/313334275' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:58.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:58:59 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:58:59] "GET /metrics HTTP/1.1" 200 233277 "" "Prometheus/2.43.0" 2024-09-10T18:58:59.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripSparseReadPP_smithi107-80070-52"}]': finished 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: osdmap e256: 8 total, 8 up, 8 in 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:58:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:58:59 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: pgmap v347: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 1.5 KiB/s wr, 4 op/s 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:00.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: osdmap e257: 8 total, 8 up, 8 in 2024-09-10T18:59:00.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:00.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:00.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]: dispatch 2024-09-10T18:59:01.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:01 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:59:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:01 smithi162 ceph-mon[33228]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:59:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:59:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[29404]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:59:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-35"}]': finished 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[40355]: osdmap e258: 8 total, 8 up, 8 in 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[29404]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:59:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[29404]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:59:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[40355]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:59:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:02 smithi107 ceph-mon[40355]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:59:02.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:02 smithi162 ceph-mon[33228]: pgmap v350: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 767 B/s wr, 1 op/s 2024-09-10T18:59:02.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripAppendPP_smithi107-80070-53", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:02.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:02 smithi162 ceph-mon[33228]: osdmap e259: 8 total, 8 up, 8 in 2024-09-10T18:59:02.677 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[29404]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[40355]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:03.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:03 smithi162 ceph-mon[33228]: osdmap e260: 8 total, 8 up, 8 in 2024-09-10T18:59:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: pgmap v353: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: pgmap v353: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: pgmap v353: 236 pgs: 32 unknown, 8 creating+peering, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-37","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: osdmap e261: 8 total, 8 up, 8 in 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: osdmap e262: 8 total, 8 up, 8 in 2024-09-10T18:59:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1264754030' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]: dispatch 2024-09-10T18:59:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:06 smithi162 ceph-mon[33228]: pgmap v357: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[29404]: pgmap v357: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripAppendPP_smithi107-80070-53"}]': finished 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: osdmap e263: 8 total, 8 up, 8 in 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:06.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:06.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:06 smithi107 ceph-mon[40355]: pgmap v357: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]': finished 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:07.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:07.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:07.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]': finished 2024-09-10T18:59:07.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:07.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-37", "mode": "writeback"}]': finished 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsCompletePP_smithi107-80070-54", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: osdmap e264: 8 total, 8 up, 8 in 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:07.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:07.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:08 smithi162 ceph-mon[33228]: pgmap v360: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:59:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:08 smithi162 ceph-mon[33228]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:59:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:08.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[29404]: pgmap v360: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:59:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[29404]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:59:08.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[40355]: pgmap v360: 228 pgs: 8 unknown, 220 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[40355]: osdmap e265: 8 total, 8 up, 8 in 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:08.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]: dispatch 2024-09-10T18:59:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:09 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:09 smithi162 ceph-mon[33228]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:59:09.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:09 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:09] "GET /metrics HTTP/1.1" 200 228483 "" "Prometheus/2.43.0" 2024-09-10T18:59:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[29404]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsCompletePP_smithi107-80070-54", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-37"}]': finished 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[40355]: osdmap e266: 8 total, 8 up, 8 in 2024-09-10T18:59:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:09 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:10 smithi162 ceph-mon[33228]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:59:10.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:10 smithi162 ceph-mon[33228]: pgmap v363: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:10 smithi107 ceph-mon[29404]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:59:10.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:10 smithi107 ceph-mon[29404]: pgmap v363: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:10 smithi107 ceph-mon[40355]: osdmap e267: 8 total, 8 up, 8 in 2024-09-10T18:59:10.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:10 smithi107 ceph-mon[40355]: pgmap v363: 204 pgs: 8 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:11.130 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:11.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:11 smithi162 ceph-mon[33228]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:59:11.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:11.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:11.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[40355]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:59:11.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:11.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[29404]: osdmap e268: 8 total, 8 up, 8 in 2024-09-10T18:59:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:11.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:12.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:12.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:12.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:12.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:12.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-39","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: osdmap e269: 8 total, 8 up, 8 in 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1863654680' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]: dispatch 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: pgmap v366: 228 pgs: 32 unknown, 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsCompletePP_smithi107-80070-54"}]': finished 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: osdmap e270: 8 total, 8 up, 8 in 2024-09-10T18:59:12.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:12.468 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:13.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-IsSafePP_smithi107-80070-55", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: osdmap e271: 8 total, 8 up, 8 in 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]: dispatch 2024-09-10T18:59:13.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:14.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:14 smithi162 ceph-mon[33228]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:14 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:14 smithi107 ceph-mon[29404]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:14 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:14 smithi107 ceph-mon[40355]: pgmap v369: 228 pgs: 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:14 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]': finished 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]': finished 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:59:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-39", "mode": "writeback"}]': finished 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: osdmap e272: 8 total, 8 up, 8 in 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "IsSafePP_smithi107-80070-55", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: osdmap e273: 8 total, 8 up, 8 in 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]: dispatch 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: pgmap v372: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:16 smithi162 ceph-mon[33228]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:59:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: pgmap v372: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[40355]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: pgmap v372: 236 pgs: 8 unknown, 228 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-39"}]': finished 2024-09-10T18:59:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:16 smithi107 ceph-mon[29404]: osdmap e274: 8 total, 8 up, 8 in 2024-09-10T18:59:17.139 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [==========] Running 77 tests from 4 test suites. 2024-09-10T18:59:17.139 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] Global test environment set-up. 2024-09-10T18:59:17.139 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP 2024-09-10T18:59:17.139 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: seed 80290 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.Dirty 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.Dirty (274 ms) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.FlushWriteRaces 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.FlushWriteRaces (12087 ms) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierPP.HitSetNone 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierPP.HitSetNone (3 ms) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 3 tests from LibRadosTierPP (12364 ms total) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Overlay 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Overlay (7102 ms) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Promote 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Promote (8034 ms) 2024-09-10T18:59:17.140 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnap 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnap (9983 ms) 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapScrub 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [4] 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [5,4] 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [6,5,4] 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: my_snaps [7,6,5,4] 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting some heads 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 7 2024-09-10T18:59:17.141 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 6 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 5 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: promoting from clones for snap 4 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapScrub (46014 ms) 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteSnapTrimRace 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteSnapTrimRace (10248 ms) 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Whiteout 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Whiteout (7748 ms) 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.WhiteoutDeleteCreate (8046 ms) 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Evict 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Evict (8031 ms) 2024-09-10T18:59:17.142 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap (10041 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnap2 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnap2 (8920 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ListSnap 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ListSnap (10044 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.EvictSnapRollbackReadRace (12979 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlush 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlush (7959 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.Flush 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.Flush (8029 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushSnap 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushSnap (13068 ms) 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.FlushTryFlushRaces 2024-09-10T18:59:17.143 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.FlushTryFlushRaces (7932 ms) 2024-09-10T18:59:17.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TryFlushReadRace 2024-09-10T18:59:17.144 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TryFlushReadRace (8025 ms) 2024-09-10T18:59:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:18 smithi162 ceph-mon[33228]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:59:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:18 smithi162 ceph-mon[33228]: pgmap v375: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[40355]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:59:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[40355]: pgmap v375: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:18.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[29404]: osdmap e275: 8 total, 8 up, 8 in 2024-09-10T18:59:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:18.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:18 smithi107 ceph-mon[29404]: pgmap v375: 196 pgs: 196 active+clean; 4.6 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:19.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:59:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:19.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:19] "GET /metrics HTTP/1.1" 200 230867 "" "Prometheus/2.43.0" 2024-09-10T18:59:19.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:19.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: osdmap e276: 8 total, 8 up, 8 in 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3858219148' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: pgmap v378: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:20.425 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: pgmap v378: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:20.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"IsSafePP_smithi107-80070-55"}]': finished 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-41","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: osdmap e277: 8 total, 8 up, 8 in 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: pgmap v378: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:20.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:21.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:21.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:21.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ReturnValuePP_smithi107-80070-56", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: osdmap e278: 8 total, 8 up, 8 in 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T18:59:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:22.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[29404]: pgmap v380: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 974 KiB/s wr, 1 op/s 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[29404]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[40355]: pgmap v380: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 974 KiB/s wr, 1 op/s 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[40355]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:59:22.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:22.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-mon[33228]: pgmap v380: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 974 KiB/s wr, 1 op/s 2024-09-10T18:59:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_count","val": "2"}]': finished 2024-09-10T18:59:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-mon[33228]: osdmap e279: 8 total, 8 up, 8 in 2024-09-10T18:59:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:23.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ReturnValuePP_smithi107-80070-56", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: osdmap e280: 8 total, 8 up, 8 in 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T18:59:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[29404]: pgmap v383: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:59:24.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[29404]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:59:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[40355]: pgmap v383: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:59:24.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:24 smithi107 ceph-mon[40355]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:59:24.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:24 smithi162 ceph-mon[33228]: pgmap v383: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:24 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-41","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T18:59:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:24 smithi162 ceph-mon[33228]: osdmap e281: 8 total, 8 up, 8 in 2024-09-10T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:59:25.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:25.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: osdmap e282: 8 total, 8 up, 8 in 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]: dispatch 2024-09-10T18:59:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]': finished 2024-09-10T18:59:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]': finished 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:26.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: pgmap v386: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-41"}]': finished 2024-09-10T18:59:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: osdmap e283: 8 total, 8 up, 8 in 2024-09-10T18:59:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1516185908' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ReturnValuePP_smithi107-80070-56"}]': finished 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: osdmap e284: 8 total, 8 up, 8 in 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:27.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:27.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:28.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: pgmap v389: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: pgmap v389: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: pgmap v389: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushPP_smithi107-80070-57", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: osdmap e285: 8 total, 8 up, 8 in 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:28.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:28.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:29] "GET /metrics HTTP/1.1" 200 230867 "" "Prometheus/2.43.0" 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:29.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:29.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:59:29.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:29.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:29.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:29.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-43","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:29.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: osdmap e286: 8 total, 8 up, 8 in 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-6","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushPP_smithi107-80070-57", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: osdmap e287: 8 total, 8 up, 8 in 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:29.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]: dispatch 2024-09-10T18:59:30.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-mon[33228]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:30.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:59:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-mon[33228]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:59:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[29404]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[29404]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[40355]: pgmap v392: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_count","val": "8"}]': finished 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[40355]: osdmap e288: 8 total, 8 up, 8 in 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T18:59:31.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:32.358 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:32.358 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: pgmap v395: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.359 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetRead 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: ok, hit_set contains 193:602f83fe:::foo:head 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetRead (9074 ms) 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetWrite 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 0 ls 1725994773,0 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 1 ls 1725994773,0 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 2 ls 1725994773,0 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 3 ls 1725994773,0 2024-09-10T18:59:32.447 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 4 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 5 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 6 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 7 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 8 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 9 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 10 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 11 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 12 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 13 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 14 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 15 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 16 ls 1725994773,0 2024-09-10T18:59:32.448 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 17 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 18 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 19 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 20 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 21 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 22 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 23 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 24 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 25 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 26 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 27 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 28 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 29 ls 1725994773,0 2024-09-10T18:59:32.449 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 30 ls 1725994773,0 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg 31 ls 1725994773,0 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: pg_num = 32 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6cac518f:::0:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:02547ec2:::1:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f905c69b:::2:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:cfc208b3:::3:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d83876eb:::4:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b29083e3:::5:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c4fdafeb:::6:head 2024-09-10T18:59:32.450 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:5c6b0b28:::7:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:bd63b0f1:::8:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e960b815:::9:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:52ea6a34:::10:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:89d3ae78:::11:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:de5d7c5f:::12:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:566253c9:::13:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:62a1935d:::14:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:863748b0:::15:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3958e169:::16:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4d4dabf9:::17:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:8391935d:::18:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:28883081:::19:head 2024-09-10T18:59:32.451 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:69259c59:::20:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4bdb80b7:::21:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a11c5d71:::22:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:271af37b:::23:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:95b121be:::24:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:58d1031b:::25:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:0a050783:::26:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c709704c:::27:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:cbe56eaf:::28:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:86b4b162:::29:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:70d89383:::30:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:dd450c7c:::31:head 2024-09-10T18:59:32.452 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6d5729b1:::32:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c388f3fb:::33:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:56cfea31:::34:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9dbc1bf7:::35:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:40b74ccd:::36:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4d5aaf42:::37:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:920f362c:::38:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6cc53222:::39:head 2024-09-10T18:59:32.453 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9cad833f:::40:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:1ea84d41:::41:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c4480ef6:::42:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a694361e:::43:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d1bd33e9:::44:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ddc2cd5d:::45:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:2b782207:::46:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7b187fca:::47:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:90ecdf6f:::48:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a5ed95fe:::49:head 2024-09-10T18:59:32.454 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ea0eaa55:::50:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f33ef17b:::51:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a0d1b2f6:::52:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:60c5229e:::53:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:edcbc575:::54:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:102cf253:::55:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:efb7fb0b:::56:head 2024-09-10T18:59:32.455 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:50d0a326:::57:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d4dc5daf:::58:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3a130462:::59:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ec87ed71:::60:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d5bc9454:::61:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3ddfe313:::62:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7c2816b9:::63:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:47e00e4d:::64:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c6410c18:::65:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b48ed237:::66:head 2024-09-10T18:59:32.456 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:cd63ad31:::67:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b179e92b:::68:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:0d9f741a:::69:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6d3352ae:::70:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c6d5c19e:::71:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:bc4729c3:::72:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:77e930b9:::73:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:0abeecfd:::74:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b7c37e15:::75:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b6378398:::76:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:02bd68de:::77:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:cc795d2d:::78:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:630d4fea:::79:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e0d29ef5:::80:head 2024-09-10T18:59:32.457 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:fd6f13d2:::81:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:606461d5:::82:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:eadbdc43:::83:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:8761d0bb:::84:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9ef0186f:::85:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e0d41294:::86:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:961de695:::87:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:1423148f:::88:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:633a8fa2:::89:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a8653809:::90:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3dac8b33:::91:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:35aad435:::92:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f6dcc343:::93:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:dbbdad87:::94:head 2024-09-10T18:59:32.458 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:1cb48ce0:::95:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:03cd461c:::96:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:17a4ea99:::97:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9993c9a7:::98:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6394211c:::99:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:94c7ae57:::100:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6fdee5bb:::101:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9a477fd1:::102:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:eb850916:::103:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:affc56b9:::104:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b42dc814:::105:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f319f8f0:::106:head 2024-09-10T18:59:32.459 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9a40b9de:::107:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:8b524f28:::108:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e3de589f:::109:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:90f90a5b:::110:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a7b4f1d7:::111:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:af51766e:::112:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b6f90bd1:::113:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e0261208:::114:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c9569ef7:::115:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:61bebe50:::116:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:fe93412b:::117:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d3d38bee:::118:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3100ba0c:::119:head 2024-09-10T18:59:32.460 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d0560ada:::120:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f0ea8b35:::121:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:766f231a:::122:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a07a2582:::123:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:bd7c6b3a:::124:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:fb2ddaff:::125:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4408e1fe:::126:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ee1df7a7:::127:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c3002909:::128:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4f48ffa9:::129:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:edf38733:::130:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c08425c0:::131:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:5f902d98:::132:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:41ea2c93:::133:head 2024-09-10T18:59:32.461 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:813cee13:::134:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:0131818d:::135:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:26ba5a85:::136:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:381b8a5a:::137:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:28797e47:::138:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:bfca7f22:::139:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:36807075:::140:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:80b03975:::141:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:5c15709b:::142:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f39ea15e:::143:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ea992956:::144:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:48887b1c:::145:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:9f24a9dd:::146:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:987f100b:::147:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d2dd3581:::148:head 2024-09-10T18:59:32.462 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7fed1808:::149:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c80b70e9:::150:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:85ed90f9:::151:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:36428b24:::152:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d044c34a:::153:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7c18bf58:::154:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d1c21232:::155:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a7a3c575:::156:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:87da0633:::157:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d5ac3822:::158:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3f20522d:::159:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6ca26563:::160:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:532ce135:::161:head 2024-09-10T18:59:32.463 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:c78863e6:::162:head 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: pgmap v395: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:59:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: pgmap v395: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_period","val": "600"}]': finished 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: osdmap e289: 8 total, 8 up, 8 in 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]: dispatch 2024-09-10T18:59:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:32.673 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:33.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:33.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:59:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-43","var": "hit_set_type","val": "explicit_hash"}]': finished 2024-09-10T18:59:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: osdmap e290: 8 total, 8 up, 8 in 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/707049984' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool get","pool":"test-rados-api-smithi107-80290-43","var": "pg_num","format": "json"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:33.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]: dispatch 2024-09-10T18:59:34.616 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]': finished 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:34.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]': finished 2024-09-10T18:59:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:59:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.618 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:34.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: pgmap v398: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushPP_smithi107-80070-57"}]': finished 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-43"}]': finished 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: osdmap e291: 8 total, 8 up, 8 in 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:35 smithi162 ceph-mon[33228]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:59:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:35 smithi162 ceph-mon[33228]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:59:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[29404]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:59:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[29404]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:59:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-FlushAsyncPP_smithi107-80070-58", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[40355]: osdmap e292: 8 total, 8 up, 8 in 2024-09-10T18:59:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:35 smithi107 ceph-mon[40355]: osdmap e293: 8 total, 8 up, 8 in 2024-09-10T18:59:36.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: pgmap v401: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: pgmap v401: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: pgmap v401: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "FlushAsyncPP_smithi107-80070-58", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-45","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: osdmap e294: 8 total, 8 up, 8 in 2024-09-10T18:59:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:36.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: pgmap v404: 236 pgs: 8 creating+peering, 8 unknown, 220 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:38 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:38.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: pgmap v404: 236 pgs: 8 creating+peering, 8 unknown, 220 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: pgmap v404: 236 pgs: 8 creating+peering, 8 unknown, 220 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: osdmap e295: 8 total, 8 up, 8 in 2024-09-10T18:59:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T18:59:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:38 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:39] "GET /metrics HTTP/1.1" 200 230848 "" "Prometheus/2.43.0" 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[40355]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[29404]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:39.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:39.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:39.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_count","val": "3"}]': finished 2024-09-10T18:59:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:39 smithi162 ceph-mon[33228]: osdmap e296: 8 total, 8 up, 8 in 2024-09-10T18:59:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: pgmap v407: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_period","val": "3"}]': finished 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: osdmap e297: 8 total, 8 up, 8 in 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T18:59:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:40 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T18:59:41.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:59:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:59:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:59:41.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:41.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:59:41.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:59:41.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2148208296' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"FlushAsyncPP_smithi107-80070-58"}]': finished 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: osdmap e298: 8 total, 8 up, 8 in 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-45","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: osdmap e299: 8 total, 8 up, 8 in 2024-09-10T18:59:41.719 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:42 smithi162 ceph-mon[33228]: pgmap v410: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:42.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:42 smithi107 ceph-mon[29404]: pgmap v410: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:42.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:42 smithi107 ceph-mon[40355]: pgmap v410: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T18:59:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:43 smithi162 ceph-mon[33228]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:59:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:43 smithi162 ceph-mon[33228]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:59:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[29404]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:59:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[29404]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:59:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[40355]: osdmap e300: 8 total, 8 up, 8 in 2024-09-10T18:59:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "RoundTripWriteFullPP_smithi107-80070-59", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:43 smithi107 ceph-mon[40355]: osdmap e301: 8 total, 8 up, 8 in 2024-09-10T18:59:44.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:44 smithi162 ceph-mon[33228]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:44 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:44 smithi162 ceph-mon[33228]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:59:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[29404]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[29404]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:59:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[40355]: pgmap v413: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:44 smithi107 ceph-mon[40355]: osdmap e302: 8 total, 8 up, 8 in 2024-09-10T18:59:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:46 smithi162 ceph-mon[33228]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:46 smithi162 ceph-mon[33228]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:59:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[29404]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[29404]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[40355]: pgmap v416: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[40355]: osdmap e303: 8 total, 8 up, 8 in 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7462ddf6:::1.RoundTripAppendPP (2986 ms) 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RacingRemovePP 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RacingRemovePP (3005 ms) 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP (3013 ms) 2024-09-10T18:59:47.284 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.RoundTripCmpExtPP2 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.RoundTripCmpExtPP2 (3246 ms) 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.PoolEIOFlag 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: setting pool EIO 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: max_success 1681, min_failed 1682 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.PoolEIOFlag (3748 ms) 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAio.MultiReads 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAio.MultiReads (3252 ms) 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 32 tests from LibRadosAio (117353 ms total) 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.ReadIntoBufferlist 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.ReadIntoBufferlist (2776 ms) 2024-09-10T18:59:47.285 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.XattrsRoundTripPP 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.XattrsRoundTripPP (9042 ms) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RmXattrPP 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RmXattrPP (14941 ms) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioPP.RemoveTestPP 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioPP.RemoveTestPP (3016 ms) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 4 tests from LibRadosAioPP (29775 ms total) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosIoPP.XattrListPP 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosIoPP.XattrListPP (3013 ms) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 1 test from LibRadosIoPP (3013 ms total) 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-10T18:59:47.286 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleWritePP 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleWritePP (13980 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.WaitForSafePP 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.WaitForSafePP (7041 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP (6951 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP2 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP2 (7024 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripPP3 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripPP3 (3009 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripSparseReadPP 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripSparseReadPP (7035 ms) 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripAppendPP 2024-09-10T18:59:47.287 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripAppendPP (6933 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsCompletePP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsCompletePP (7035 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.IsSafePP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.IsSafePP (7023 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ReturnValuePP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ReturnValuePP (7065 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushPP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushPP (7029 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.FlushAsyncPP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.FlushAsyncPP (7020 ms) 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullPP 2024-09-10T18:59:47.288 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP (7018 ms) 2024-09-10T18:59:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:47 smithi162 ceph-mon[33228]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:59:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:47 smithi162 ceph-mon[33228]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:59:47.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[29404]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:59:47.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[29404]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[40355]: osdmap e304: 8 total, 8 up, 8 in 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]: dispatch 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2933079565' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"RoundTripWriteFullPP_smithi107-80070-59"}]': finished 2024-09-10T18:59:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:47 smithi107 ceph-mon[40355]: osdmap e305: 8 total, 8 up, 8 in 2024-09-10T18:59:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:48 smithi162 ceph-mon[33228]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:48 smithi107 ceph-mon[29404]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:48 smithi107 ceph-mon[40355]: pgmap v419: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:49] "GET /metrics HTTP/1.1" 200 228482 "" "Prometheus/2.43.0" 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[29404]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2921520410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[40355]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2921520410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:49.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:49 smithi162 ceph-mon[33228]: osdmap e306: 8 total, 8 up, 8 in 2024-09-10T18:59:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2921520410' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:50 smithi162 ceph-mon[33228]: pgmap v422: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:59:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:50 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:50 smithi162 ceph-mon[33228]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:59:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[40355]: pgmap v422: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:59:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[40355]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:59:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[29404]: pgmap v422: 260 pgs: 32 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T18:59:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "RoundTripWriteFullPP2_smithi107-80070-60","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:50 smithi107 ceph-mon[29404]: osdmap e307: 8 total, 8 up, 8 in 2024-09-10T18:59:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 18:59:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T18:59:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:51 smithi162 ceph-mon[33228]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:59:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[29404]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:59:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[40355]: osdmap e308: 8 total, 8 up, 8 in 2024-09-10T18:59:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:52 smithi162 ceph-mon[33228]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-10T18:59:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:52 smithi162 ceph-mon[33228]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:59:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:52.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 18:59:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T18:59:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[29404]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-10T18:59:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[29404]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:59:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[40355]: pgmap v425: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s wr, 1 op/s 2024-09-10T18:59:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPP_smithi107-80070-61", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[40355]: osdmap e309: 8 total, 8 up, 8 in 2024-09-10T18:59:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:53.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]': finished 2024-09-10T18:59:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:53 smithi162 ceph-mon[33228]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:59:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:59:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]': finished 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[29404]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: osdmap e310: 8 total, 8 up, 8 in 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]: dispatch 2024-09-10T18:59:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPP_smithi107-80070-61", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-45"}]': finished 2024-09-10T18:59:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:53 smithi107 ceph-mon[40355]: osdmap e311: 8 total, 8 up, 8 in 2024-09-10T18:59:54.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:54 smithi162 ceph-mon[33228]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:59:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:54 smithi162 ceph-mon[33228]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:59:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:54 smithi107 ceph-mon[29404]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:59:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:54 smithi107 ceph-mon[29404]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:59:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:54 smithi107 ceph-mon[40355]: pgmap v428: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 2 op/s 2024-09-10T18:59:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:54 smithi107 ceph-mon[40355]: osdmap e312: 8 total, 8 up, 8 in 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: pgmap v431: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:56 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: pgmap v431: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:59:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: pgmap v431: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: osdmap e313: 8 total, 8 up, 8 in 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:56 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:57 smithi162 ceph-mon[33228]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:59:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:59:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[29404]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-47","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: osdmap e314: 8 total, 8 up, 8 in 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3019271524' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPP_smithi107-80070-61"}]': finished 2024-09-10T18:59:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T18:59:57.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:57 smithi107 ceph-mon[40355]: osdmap e315: 8 total, 8 up, 8 in 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: pgmap v434: 228 pgs: 7 unknown, 221 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: pgmap v434: 228 pgs: 7 unknown, 221 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: pgmap v434: 228 pgs: 7 unknown, 221 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T18:59:58.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: osdmap e316: 8 total, 8 up, 8 in 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]: dispatch 2024-09-10T18:59:58.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T18:59:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 18:59:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:18:59:59] "GET /metrics HTTP/1.1" 200 228482 "" "Prometheus/2.43.0" 2024-09-10T18:59:59.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 18:59:59 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 18:59:59 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T18:59:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 18:59:59 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]': finished 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: pool 'test-rados-api-smithi107-80290-47' with cache_mode writeback needs hit_set_type to be set but it is not 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:00:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]': finished 2024-09-10T19:00:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: pool 'test-rados-api-smithi107-80290-47' with cache_mode writeback needs hit_set_type to be set but it is not 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: pgmap v437: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-47", "mode": "writeback"}]': finished 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: osdmap e317: 8 total, 8 up, 8 in 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: Health detail: HEALTH_WARN 1 cache pools are missing hit_sets; 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: [WRN] CACHE_POOL_NO_HIT_SET: 1 cache pools are missing hit_sets 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: pool 'test-rados-api-smithi107-80290-47' with cache_mode writeback needs hit_set_type to be set but it is not 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: [WRN] POOL_APP_NOT_ENABLED: 1 pool(s) do not have an application enabled 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:00:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:00:00.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "SimpleStatPPNS_smithi107-80070-62", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:00.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:00.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: osdmap e318: 8 total, 8 up, 8 in 2024-09-10T19:00:00.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:00.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.382 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: pgmap v440: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 255 B/s wr, 0 op/s 2024-09-10T19:00:02.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:02.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: osdmap e319: 8 total, 8 up, 8 in 2024-09-10T19:00:02.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:02.673 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: osdmap e320: 8 total, 8 up, 8 in 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:03.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:03.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T19:00:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: pgmap v443: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: osdmap e321: 8 total, 8 up, 8 in 2024-09-10T19:00:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1226434515' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]: dispatch 2024-09-10T19:00:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:00:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"SimpleStatPPNS_smithi107-80070-62"}]': finished 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: osdmap e322: 8 total, 8 up, 8 in 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:00:05.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:05.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: pgmap v446: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-47","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-StatRemovePP_smithi107-80070-63", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: osdmap e323: 8 total, 8 up, 8 in 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:07.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:07 smithi162 ceph-mon[33228]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T19:00:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[29404]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T19:00:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[40355]: osdmap e324: 8 total, 8 up, 8 in 2024-09-10T19:00:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: pgmap v449: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "StatRemovePP_smithi107-80070-63", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]': finished 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: osdmap e325: 8 total, 8 up, 8 in 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:08.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:08.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-47"}]: dispatch 2024-09-10T19:00:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:09] "GET /metrics HTTP/1.1" 200 230899 "" "Prometheus/2.43.0" 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: osdmap e326: 8 total, 8 up, 8 in 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: osdmap e327: 8 total, 8 up, 8 in 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:10.923 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: pgmap v452: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: pgmap v452: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: pgmap v452: 204 pgs: 8 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-49","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: osdmap e328: 8 total, 8 up, 8 in 2024-09-10T19:00:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]: dispatch 2024-09-10T19:00:10.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:10.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:12.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: pgmap v455: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:12.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: pgmap v455: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:00:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: pgmap v455: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 511 B/s rd, 0 op/s 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/913281079' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"StatRemovePP_smithi107-80070-63"}]': finished 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: osdmap e329: 8 total, 8 up, 8 in 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:13.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:13.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:13.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:13.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-ExecuteClassPP_smithi107-80070-64", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: osdmap e330: 8 total, 8 up, 8 in 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:13.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]: dispatch 2024-09-10T19:00:13.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:14.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:14 smithi162 ceph-mon[33228]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:14 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]': finished 2024-09-10T19:00:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:14 smithi162 ceph-mon[33228]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[29404]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]': finished 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[29404]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[40355]: pgmap v458: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-49", "mode": "readproxy"}]': finished 2024-09-10T19:00:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:14 smithi107 ceph-mon[40355]: osdmap e331: 8 total, 8 up, 8 in 2024-09-10T19:00:15.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:15 smithi162 ceph-mon[33228]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T19:00:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:15 smithi107 ceph-mon[29404]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T19:00:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "ExecuteClassPP_smithi107-80070-64", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:15 smithi107 ceph-mon[40355]: osdmap e332: 8 total, 8 up, 8 in 2024-09-10T19:00:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:16 smithi162 ceph-mon[33228]: pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:16 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:16 smithi162 ceph-mon[33228]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T19:00:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[29404]: pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[29404]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T19:00:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[40355]: pgmap v461: 236 pgs: 8 unknown, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:00:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[40355]: osdmap e333: 8 total, 8 up, 8 in 2024-09-10T19:00:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[29404]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T19:00:17.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:17.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[40355]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T19:00:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:17.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:17 smithi162 ceph-mon[33228]: osdmap e334: 8 total, 8 up, 8 in 2024-09-10T19:00:17.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[29404]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[29404]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[40355]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[40355]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:18 smithi162 ceph-mon[33228]: pgmap v464: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:18 smithi162 ceph-mon[33228]: osdmap e335: 8 total, 8 up, 8 in 2024-09-10T19:00:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1155623993' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]: dispatch 2024-09-10T19:00:19.438 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:19] "GET /metrics HTTP/1.1" 200 230897 "" "Prometheus/2.43.0" 2024-09-10T19:00:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[29404]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[40355]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"ExecuteClassPP_smithi107-80070-64"}]': finished 2024-09-10T19:00:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:19 smithi162 ceph-mon[33228]: osdmap e336: 8 total, 8 up, 8 in 2024-09-10T19:00:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[29404]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[29404]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T19:00:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[40355]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[40355]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T19:00:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:20 smithi162 ceph-mon[33228]: pgmap v467: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 1 op/s 2024-09-10T19:00:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-OmapPP_smithi107-80070-65", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:20 smithi162 ceph-mon[33228]: osdmap e337: 8 total, 8 up, 8 in 2024-09-10T19:00:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[29404]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T19:00:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[29404]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[29404]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[40355]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[40355]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:21 smithi107 ceph-mon[40355]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T19:00:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:21 smithi162 ceph-mon[33228]: osdmap e338: 8 total, 8 up, 8 in 2024-09-10T19:00:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:21 smithi162 ceph-mon[33228]: pgmap v470: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "OmapPP_smithi107-80070-65", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:21 smithi162 ceph-mon[33228]: osdmap e339: 8 total, 8 up, 8 in 2024-09-10T19:00:22.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:23 smithi107 ceph-mon[29404]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T19:00:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:23 smithi107 ceph-mon[40355]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T19:00:23.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:23 smithi162 ceph-mon[33228]: osdmap e340: 8 total, 8 up, 8 in 2024-09-10T19:00:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:24.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T19:00:24.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:24.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:24.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: pgmap v473: 236 pgs: 8 creating+peering, 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: osdmap e341: 8 total, 8 up, 8 in 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: osdmap e342: 8 total, 8 up, 8 in 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]: dispatch 2024-09-10T19:00:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:25 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:25 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:25.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:25.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:25 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:26.478 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.RoundTripWriteFullP63:head 2024-09-10T19:00:26.478 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:5d165639:::164:head 2024-09-10T19:00:26.478 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f43765fc:::165:head 2024-09-10T19:00:26.478 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b4c720e9:::166:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e694b040:::167:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:afa38db2:::168:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:77ba9f53:::169:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:87495034:::170:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7c96bf0e:::171:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:dbe346cc:::172:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:e943ec24:::173:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f97a9c0c:::174:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:6f26e74d:::175:head 2024-09-10T19:00:26.479 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4f95e106:::176:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:0e6f2f8f:::177:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:05db05f1:::178:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:38a78d66:::179:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:d095610b:::180:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:a1a9d709:::181:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:1e5d39db:::182:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:f7df4fb9:::183:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:03a7f161:::184:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ba70721e:::185:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:28e5662d:::186:head 2024-09-10T19:00:26.480 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:973d52de:::187:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:4303eb1c:::188:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:b990b48e:::189:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:29b8165b:::190:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:3547f197:::191:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:7e260936:::192:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:1abec7b1:::193:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:10fdda93:::194:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:15817eea:::195:head 2024-09-10T19:00:26.481 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:770bab57:::196:head 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:ed9e13e7:::197:head 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:71471a8f:::198:head 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: checking for 195:10fb1d02:::199:head 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetWrite (8027 ms) 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HitSetTrim 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,0 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first is 1725994782 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,0 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,0 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,0 2024-09-10T19:00:26.482 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,1725994790,1725994791,0 2024-09-10T19:00:26.483 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,1725994790,1725994791,0 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994782,1725994784,1725994785,1725994787,1725994788,1725994790,1725994791,0 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725994785,1725994787,1725994788,1725994790,1725994791,1725994793,1725994794,0 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first now 1725994785, trimmed 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HitSetTrim (20090 ms) 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PromoteOn2ndRead 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: foo0 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PromoteOn2ndRead (14054 ms) 2024-09-10T19:00:26.484 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ProxyRead 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:26.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: pgmap v476: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/147019371' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"OmapPP_smithi107-80070-65"}]': finished 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]': finished 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: osdmap e343: 8 total, 8 up, 8 in 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:26.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:26.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-49"}]: dispatch 2024-09-10T19:00:28.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:28.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:27 smithi162 ceph-mon[33228]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T19:00:28.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:28.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:28.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[29404]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T19:00:28.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:28.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-MultiWritePP_smithi107-80070-66", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:00:28.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[40355]: osdmap e344: 8 total, 8 up, 8 in 2024-09-10T19:00:28.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:29.065 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: pgmap v479: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: pgmap v479: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:29.066 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: pgmap v479: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: osdmap e345: 8 total, 8 up, 8 in 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "MultiWritePP_smithi107-80070-66", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-51","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: osdmap e346: 8 total, 8 up, 8 in 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:00:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:29] "GET /metrics HTTP/1.1" 200 230897 "" "Prometheus/2.43.0" 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: pgmap v482: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:30.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:30.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: pgmap v482: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:30.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:30.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T19:00:30.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: pgmap v482: 236 pgs: 40 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: osdmap e347: 8 total, 8 up, 8 in 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:30.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: osdmap e348: 8 total, 8 up, 8 in 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]: dispatch 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:31.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:32.501 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ P2 2024-09-10T19:00:32.501 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.RoundTripWriteFullPP2 (3035 ms) 2024-09-10T19:00:32.501 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPP 2024-09-10T19:00:32.501 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPP (7017 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.SimpleStatPPNS 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.SimpleStatPPNS (7023 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.StatRemovePP 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.StatRemovePP (7037 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.ExecuteClassPP 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.ExecuteClassPP (7040 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.OmapPP 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.OmapPP (7029 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ RUN ] LibRadosAioEC.MultiWritePP 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ OK ] LibRadosAioEC.MultiWritePP (7033 ms) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] 20 tests from LibRadosAioEC (139378 ms total) 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: 2024-09-10T19:00:32.502 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [----------] Global test environment tear-down 2024-09-10T19:00:32.503 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [==========] 57 tests from 4 test suites ran. (289519 ms total) 2024-09-10T19:00:32.503 INFO:tasks.workunit.client.0.smithi107.stdout: api_aio_pp: [ PASSED ] 57 tests. 2024-09-10T19:00:32.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]': finished 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T19:00:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:32.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:32.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]': finished 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: pgmap v485: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-51", "mode": "writeback"}]': finished 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: osdmap e349: 8 total, 8 up, 8 in 2024-09-10T19:00:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]: dispatch 2024-09-10T19:00:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:32.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:33.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:33.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:33.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/587873138' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"MultiWritePP_smithi107-80070-66"}]': finished 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: osdmap e350: 8 total, 8 up, 8 in 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:33.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[29404]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[29404]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[40355]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[40355]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T19:00:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:34.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:34 smithi162 ceph-mon[33228]: pgmap v488: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:00:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:34 smithi162 ceph-mon[33228]: osdmap e351: 8 total, 8 up, 8 in 2024-09-10T19:00:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:35.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:35.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:00:35.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:00:35.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: osdmap e352: 8 total, 8 up, 8 in 2024-09-10T19:00:35.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:00:35.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:00:35.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: osdmap e353: 8 total, 8 up, 8 in 2024-09-10T19:00:35.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:35.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:00:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:36 smithi162 ceph-mon[33228]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:00:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:36 smithi162 ceph-mon[33228]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T19:00:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[29404]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:00:36.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[29404]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T19:00:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[40355]: pgmap v491: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:00:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-51","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:00:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:36 smithi107 ceph-mon[40355]: osdmap e354: 8 total, 8 up, 8 in 2024-09-10T19:00:37.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:37 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:00:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:37 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:00:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:37 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:00:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:38 smithi162 ceph-mon[33228]: pgmap v494: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:38.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:38 smithi107 ceph-mon[29404]: pgmap v494: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:38.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:38 smithi107 ceph-mon[40355]: pgmap v494: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:39] "GET /metrics HTTP/1.1" 200 228477 "" "Prometheus/2.43.0" 2024-09-10T19:00:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:40 smithi162 ceph-mon[33228]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:40.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:40 smithi107 ceph-mon[29404]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:40.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:40 smithi107 ceph-mon[40355]: pgmap v495: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 904 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:41.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:41.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:41.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:41.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:00:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:00:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:00:41.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:42.825 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:42 smithi162 ceph-mon[33228]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:00:42.825 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:42 smithi107 ceph-mon[29404]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:00:42.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:42 smithi107 ceph-mon[40355]: pgmap v496: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 769 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:00:43.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:44.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:44 smithi162 ceph-mon[33228]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:44 smithi107 ceph-mon[29404]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:44 smithi107 ceph-mon[40355]: pgmap v497: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:46.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:46 smithi162 ceph-mon[33228]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[29404]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[40355]: pgmap v498: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:00:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:00:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:48.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:48.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:48 smithi162 ceph-mon[33228]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T19:00:48.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:48 smithi162 ceph-mon[33228]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:48.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:48.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[29404]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T19:00:48.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[29404]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[40355]: osdmap e355: 8 total, 8 up, 8 in 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:48.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:48 smithi107 ceph-mon[40355]: pgmap v500: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:00:49.351 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:49] "GET /metrics HTTP/1.1" 200 228559 "" "Prometheus/2.43.0" 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T19:00:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]': finished 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: osdmap e356: 8 total, 8 up, 8 in 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:49.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-51"}]: dispatch 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T19:00:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: osdmap e357: 8 total, 8 up, 8 in 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: pgmap v503: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: osdmap e358: 8 total, 8 up, 8 in 2024-09-10T19:00:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:51.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:00:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-mon[33228]: pgmap v506: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T19:00:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[40355]: pgmap v506: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-53","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: osdmap e359: 8 total, 8 up, 8 in 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-53"}]: dispatch 2024-09-10T19:00:52.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:52 smithi107 ceph-mon[29404]: pgmap v506: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:00:52.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:00:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:00:53.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:53 smithi162 ceph-mon[33228]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T19:00:53.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:53.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:53 smithi107 ceph-mon[29404]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T19:00:53.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:53.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:53 smithi107 ceph-mon[40355]: osdmap e360: 8 total, 8 up, 8 in 2024-09-10T19:00:53.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:00:54.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:54 smithi162 ceph-mon[33228]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T19:00:54.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:54 smithi162 ceph-mon[33228]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:54.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[29404]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T19:00:54.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[29404]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[40355]: osdmap e361: 8 total, 8 up, 8 in 2024-09-10T19:00:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:54.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:54 smithi107 ceph-mon[40355]: pgmap v509: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:55.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T19:00:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-55","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: osdmap e362: 8 total, 8 up, 8 in 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-55"}]: dispatch 2024-09-10T19:00:56.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:56 smithi162 ceph-mon[33228]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T19:00:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:56 smithi162 ceph-mon[33228]: pgmap v512: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:56 smithi162 ceph-mon[33228]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T19:00:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[29404]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T19:00:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[29404]: pgmap v512: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[29404]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T19:00:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[40355]: osdmap e363: 8 total, 8 up, 8 in 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[40355]: pgmap v512: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[40355]: osdmap e364: 8 total, 8 up, 8 in 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: pgmap v515: 228 pgs: 1 active, 1 creating+activating, 21 creating+peering, 205 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:58.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T19:00:58.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: pgmap v515: 228 pgs: 1 active, 1 creating+activating, 21 creating+peering, 205 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-57","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: osdmap e365: 8 total, 8 up, 8 in 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: pgmap v515: 228 pgs: 1 active, 1 creating+activating, 21 creating+peering, 205 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:58.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-57"}]: dispatch 2024-09-10T19:00:59.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:00:59 smithi162 ceph-mon[33228]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T19:00:59.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:00:59 smithi107 ceph-mon[29404]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T19:00:59.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:00:59 smithi107 ceph-mon[40355]: osdmap e366: 8 total, 8 up, 8 in 2024-09-10T19:00:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:00:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:00:59] "GET /metrics HTTP/1.1" 200 228559 "" "Prometheus/2.43.0" 2024-09-10T19:01:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:00 smithi162 ceph-mon[33228]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T19:01:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:00 smithi162 ceph-mon[33228]: pgmap v518: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:01:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[29404]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T19:01:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[29404]: pgmap v518: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:01:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[40355]: osdmap e367: 8 total, 8 up, 8 in 2024-09-10T19:01:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:00.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:00 smithi107 ceph-mon[40355]: pgmap v518: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1023 B/s wr, 2 op/s 2024-09-10T19:01:01.168 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:01.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T19:01:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:01.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-59","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: osdmap e368: 8 total, 8 up, 8 in 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:01.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:01.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-59"}]: dispatch 2024-09-10T19:01:01.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:01.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:02.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:02 smithi107 ceph-mon[29404]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T19:01:02.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:02 smithi107 ceph-mon[29404]: pgmap v521: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:02 smithi107 ceph-mon[40355]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T19:01:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:02 smithi107 ceph-mon[40355]: pgmap v521: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:02.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:02 smithi162 ceph-mon[33228]: osdmap e369: 8 total, 8 up, 8 in 2024-09-10T19:01:02.447 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:02 smithi162 ceph-mon[33228]: pgmap v521: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:02.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[29404]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T19:01:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[40355]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T19:01:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:03 smithi162 ceph-mon[33228]: osdmap e370: 8 total, 8 up, 8 in 2024-09-10T19:01:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T19:01:04.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:04.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: pgmap v523: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-61","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: osdmap e371: 8 total, 8 up, 8 in 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T19:01:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:05.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: osdmap e372: 8 total, 8 up, 8 in 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-61"}]: dispatch 2024-09-10T19:01:06.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[29404]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:06.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:06.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[29404]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T19:01:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[40355]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:06.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:06 smithi107 ceph-mon[40355]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T19:01:06.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:06 smithi162 ceph-mon[33228]: pgmap v526: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:01:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:06 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:01:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:06 smithi162 ceph-mon[33228]: osdmap e373: 8 total, 8 up, 8 in 2024-09-10T19:01:07.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:07 smithi162 ceph-mon[33228]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T19:01:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[29404]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T19:01:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:07.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:07.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[40355]: osdmap e374: 8 total, 8 up, 8 in 2024-09-10T19:01:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:07.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: pgmap v529: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: pgmap v529: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: pgmap v529: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-63","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: osdmap e375: 8 total, 8 up, 8 in 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:09.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:09.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:09 smithi107 ceph-mon[40355]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T19:01:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:09 smithi107 ceph-mon[29404]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T19:01:09.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:09] "GET /metrics HTTP/1.1" 200 228513 "" "Prometheus/2.43.0" 2024-09-10T19:01:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:09 smithi162 ceph-mon[33228]: osdmap e376: 8 total, 8 up, 8 in 2024-09-10T19:01:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:10 smithi162 ceph-mon[33228]: pgmap v532: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:10 smithi162 ceph-mon[33228]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T19:01:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:10 smithi107 ceph-mon[29404]: pgmap v532: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:10 smithi107 ceph-mon[29404]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T19:01:10.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:10 smithi107 ceph-mon[40355]: pgmap v532: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:01:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:10 smithi107 ceph-mon[40355]: osdmap e377: 8 total, 8 up, 8 in 2024-09-10T19:01:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:11.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:11 smithi162 ceph-mon[33228]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T19:01:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:11 smithi107 ceph-mon[29404]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T19:01:11.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:11 smithi107 ceph-mon[40355]: osdmap e378: 8 total, 8 up, 8 in 2024-09-10T19:01:12.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:12 smithi162 ceph-mon[33228]: pgmap v535: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:12.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:12 smithi162 ceph-mon[33228]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T19:01:12.575 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:12 smithi107 ceph-mon[29404]: pgmap v535: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:12 smithi107 ceph-mon[29404]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T19:01:12.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:12 smithi107 ceph-mon[40355]: pgmap v535: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:12 smithi107 ceph-mon[40355]: osdmap e379: 8 total, 8 up, 8 in 2024-09-10T19:01:13.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:13 smithi162 ceph-mon[33228]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T19:01:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:13 smithi107 ceph-mon[29404]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T19:01:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:13 smithi107 ceph-mon[40355]: osdmap e380: 8 total, 8 up, 8 in 2024-09-10T19:01:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:14 smithi162 ceph-mon[33228]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T19:01:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:14 smithi107 ceph-mon[29404]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T19:01:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:14 smithi107 ceph-mon[40355]: pgmap v538: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.7 KiB/s rd, 2.7 KiB/s wr, 7 op/s 2024-09-10T19:01:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:16 smithi162 ceph-mon[33228]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T19:01:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:16 smithi107 ceph-mon[29404]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T19:01:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:16 smithi107 ceph-mon[40355]: pgmap v539: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 1.9 KiB/s wr, 5 op/s 2024-09-10T19:01:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:18 smithi162 ceph-mon[33228]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-10T19:01:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:18 smithi107 ceph-mon[29404]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-10T19:01:18.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:18 smithi107 ceph-mon[40355]: pgmap v540: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.4 KiB/s rd, 1.6 KiB/s wr, 5 op/s 2024-09-10T19:01:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:19] "GET /metrics HTTP/1.1" 200 228586 "" "Prometheus/2.43.0" 2024-09-10T19:01:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:20 smithi162 ceph-mon[33228]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T19:01:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:20 smithi162 ceph-mon[33228]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T19:01:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:20 smithi107 ceph-mon[29404]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T19:01:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:20 smithi107 ceph-mon[29404]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T19:01:20.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:20 smithi107 ceph-mon[40355]: osdmap e381: 8 total, 8 up, 8 in 2024-09-10T19:01:20.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:20 smithi107 ceph-mon[40355]: pgmap v542: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.1 KiB/s rd, 1.4 KiB/s wr, 4 op/s 2024-09-10T19:01:21.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:22.673 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:22 smithi162 ceph-mon[33228]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 688 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:01:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:22 smithi107 ceph-mon[29404]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 688 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:01:22.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:22 smithi107 ceph-mon[40355]: pgmap v543: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 688 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:01:23.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:24 smithi162 ceph-mon[33228]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:24 smithi162 ceph-mon[33228]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T19:01:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:24 smithi107 ceph-mon[29404]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:24 smithi107 ceph-mon[29404]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T19:01:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:24 smithi107 ceph-mon[40355]: pgmap v544: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:24 smithi107 ceph-mon[40355]: osdmap e382: 8 total, 8 up, 8 in 2024-09-10T19:01:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:26 smithi162 ceph-mon[33228]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:26 smithi107 ceph-mon[29404]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:26.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:26 smithi107 ceph-mon[40355]: pgmap v546: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:28 smithi162 ceph-mon[33228]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:28 smithi107 ceph-mon[29404]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:28.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:28 smithi107 ceph-mon[40355]: pgmap v547: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:29] "GET /metrics HTTP/1.1" 200 228586 "" "Prometheus/2.43.0" 2024-09-10T19:01:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:30 smithi162 ceph-mon[33228]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T19:01:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:30 smithi162 ceph-mon[33228]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:30 smithi107 ceph-mon[29404]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T19:01:30.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:30 smithi107 ceph-mon[29404]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:30 smithi107 ceph-mon[40355]: osdmap e383: 8 total, 8 up, 8 in 2024-09-10T19:01:30.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:30 smithi107 ceph-mon[40355]: pgmap v549: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:31.118 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:31.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:31.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:32.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:32 smithi162 ceph-mon[33228]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:32 smithi107 ceph-mon[29404]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:32 smithi107 ceph-mon[40355]: pgmap v550: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:32.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:33.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:33.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:34 smithi162 ceph-mon[33228]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:34.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:34 smithi107 ceph-mon[29404]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:34.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:34 smithi107 ceph-mon[40355]: pgmap v551: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:01:35.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:35 smithi162 ceph-mon[33228]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T19:01:35.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:35 smithi107 ceph-mon[29404]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T19:01:35.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:35 smithi107 ceph-mon[40355]: osdmap e384: 8 total, 8 up, 8 in 2024-09-10T19:01:36.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:36 smithi162 ceph-mon[33228]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:36.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:36 smithi107 ceph-mon[29404]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:36.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:36 smithi107 ceph-mon[40355]: pgmap v553: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:01:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:38 smithi162 ceph-mon[33228]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:38 smithi107 ceph-mon[29404]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:38 smithi107 ceph-mon[40355]: pgmap v554: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:39] "GET /metrics HTTP/1.1" 200 228532 "" "Prometheus/2.43.0" 2024-09-10T19:01:40.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:40 smithi162 ceph-mon[33228]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T19:01:40.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:40 smithi162 ceph-mon[33228]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:40.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:40 smithi107 ceph-mon[29404]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T19:01:40.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:40 smithi107 ceph-mon[29404]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:40.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:40 smithi107 ceph-mon[40355]: osdmap e385: 8 total, 8 up, 8 in 2024-09-10T19:01:40.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:40 smithi107 ceph-mon[40355]: pgmap v556: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1 op/s 2024-09-10T19:01:41.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:41.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:41 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:41.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:41 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:41 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:01:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:42 smithi162 ceph-mon[33228]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T19:01:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:42.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:42.575 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[29404]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[40355]: pgmap v557: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 767 B/s rd, 0 op/s 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:01:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:01:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:44 smithi162 ceph-mon[33228]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[29404]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[40355]: pgmap v558: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:01:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-63"}]: dispatch 2024-09-10T19:01:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:45 smithi162 ceph-mon[33228]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T19:01:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:45 smithi162 ceph-mon[33228]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T19:01:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[29404]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[29404]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[40355]: osdmap e386: 8 total, 8 up, 8 in 2024-09-10T19:01:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[40355]: osdmap e387: 8 total, 8 up, 8 in 2024-09-10T19:01:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: pgmap v560: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-65","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: osdmap e388: 8 total, 8 up, 8 in 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:01:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:48 smithi162 ceph-mon[33228]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:48 smithi162 ceph-mon[33228]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[29404]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[29404]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[40355]: pgmap v563: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:01:48.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:48 smithi107 ceph-mon[40355]: osdmap e389: 8 total, 8 up, 8 in 2024-09-10T19:01:49.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:49 smithi107 ceph-mon[29404]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T19:01:49.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:49 smithi107 ceph-mon[40355]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T19:01:49.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:49] "GET /metrics HTTP/1.1" 200 228496 "" "Prometheus/2.43.0" 2024-09-10T19:01:49.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:49 smithi162 ceph-mon[33228]: osdmap e390: 8 total, 8 up, 8 in 2024-09-10T19:01:50.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:50 smithi162 ceph-mon[33228]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:50 smithi162 ceph-mon[33228]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T19:01:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:50 smithi107 ceph-mon[29404]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:50 smithi107 ceph-mon[29404]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T19:01:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:50 smithi107 ceph-mon[40355]: pgmap v566: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:01:50.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:50 smithi107 ceph-mon[40355]: osdmap e391: 8 total, 8 up, 8 in 2024-09-10T19:01:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:01:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:01:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:51 smithi162 ceph-mon[33228]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T19:01:51.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:51 smithi107 ceph-mon[40355]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T19:01:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:51 smithi107 ceph-mon[29404]: osdmap e392: 8 total, 8 up, 8 in 2024-09-10T19:01:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:52 smithi162 ceph-mon[33228]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:52.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:01:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:01:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:52 smithi107 ceph-mon[29404]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:52.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:52 smithi107 ceph-mon[40355]: pgmap v569: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:01:53.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:53.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:01:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:54 smithi162 ceph-mon[33228]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:54 smithi162 ceph-mon[33228]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T19:01:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:54 smithi107 ceph-mon[29404]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:54 smithi107 ceph-mon[29404]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T19:01:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:54 smithi107 ceph-mon[40355]: pgmap v570: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:54 smithi107 ceph-mon[40355]: osdmap e393: 8 total, 8 up, 8 in 2024-09-10T19:01:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:56 smithi162 ceph-mon[33228]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:56 smithi107 ceph-mon[29404]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:56 smithi107 ceph-mon[40355]: pgmap v572: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.7 KiB/s wr, 6 op/s 2024-09-10T19:01:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:01:58 smithi162 ceph-mon[33228]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T19:01:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:01:58 smithi107 ceph-mon[29404]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T19:01:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:01:58 smithi107 ceph-mon[40355]: pgmap v573: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 1.3 KiB/s wr, 5 op/s 2024-09-10T19:01:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:01:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:01:59] "GET /metrics HTTP/1.1" 200 228496 "" "Prometheus/2.43.0" 2024-09-10T19:02:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-mon[33228]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T19:02:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:00.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[40355]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T19:02:00.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[29404]: pgmap v574: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1.1 KiB/s wr, 5 op/s 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-65"}]: dispatch 2024-09-10T19:02:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:01.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:02 smithi162 ceph-mon[33228]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:02:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:02 smithi162 ceph-mon[33228]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T19:02:02.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:02.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:02 smithi107 ceph-mon[40355]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:02:02.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:02 smithi107 ceph-mon[40355]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T19:02:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:02 smithi107 ceph-mon[29404]: pgmap v575: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:02:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:02 smithi107 ceph-mon[29404]: osdmap e394: 8 total, 8 up, 8 in 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:03 smithi162 ceph-mon[33228]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[29404]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: osdmap e395: 8 total, 8 up, 8 in 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-67","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:03 smithi107 ceph-mon[40355]: osdmap e396: 8 total, 8 up, 8 in 2024-09-10T19:02:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:04 smithi162 ceph-mon[33228]: pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:04 smithi162 ceph-mon[33228]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T19:02:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[40355]: pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[40355]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T19:02:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[29404]: pgmap v578: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:04 smithi107 ceph-mon[29404]: osdmap e397: 8 total, 8 up, 8 in 2024-09-10T19:02:05.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:05.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:05.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-67"}]: dispatch 2024-09-10T19:02:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:06 smithi162 ceph-mon[33228]: pgmap v581: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:06 smithi162 ceph-mon[33228]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T19:02:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:06 smithi162 ceph-mon[33228]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T19:02:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[29404]: pgmap v581: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[29404]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T19:02:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[29404]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[40355]: pgmap v581: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[40355]: osdmap e398: 8 total, 8 up, 8 in 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[40355]: osdmap e399: 8 total, 8 up, 8 in 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: pgmap v584: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-69","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: osdmap e400: 8 total, 8 up, 8 in 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:08.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-69"}]: dispatch 2024-09-10T19:02:09.395 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:09] "GET /metrics HTTP/1.1" 200 228580 "" "Prometheus/2.43.0" 2024-09-10T19:02:09.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:09 smithi162 ceph-mon[33228]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T19:02:09.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:09 smithi107 ceph-mon[29404]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T19:02:09.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:09 smithi107 ceph-mon[40355]: osdmap e401: 8 total, 8 up, 8 in 2024-09-10T19:02:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-mon[33228]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-mon[33228]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T19:02:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[29404]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[29404]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T19:02:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[40355]: pgmap v587: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[40355]: osdmap e402: 8 total, 8 up, 8 in 2024-09-10T19:02:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:11 smithi162 ceph-mon[33228]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T19:02:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[29404]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T19:02:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-71","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[40355]: osdmap e403: 8 total, 8 up, 8 in 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:02:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[29404]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:02:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[29404]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T19:02:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[40355]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:02:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:12 smithi107 ceph-mon[40355]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T19:02:12.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:12 smithi162 ceph-mon[33228]: pgmap v590: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:02:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:12 smithi162 ceph-mon[33228]: osdmap e404: 8 total, 8 up, 8 in 2024-09-10T19:02:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:13 smithi107 ceph-mon[29404]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T19:02:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:13 smithi107 ceph-mon[40355]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T19:02:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:13.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:13 smithi162 ceph-mon[33228]: osdmap e405: 8 total, 8 up, 8 in 2024-09-10T19:02:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:14 smithi107 ceph-mon[29404]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T19:02:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:14 smithi107 ceph-mon[40355]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T19:02:14.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:14 smithi162 ceph-mon[33228]: pgmap v593: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 3.0 KiB/s wr, 5 op/s 2024-09-10T19:02:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:16 smithi107 ceph-mon[29404]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-09-10T19:02:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:16 smithi107 ceph-mon[40355]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-09-10T19:02:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:16.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:16 smithi162 ceph-mon[33228]: pgmap v594: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.1 KiB/s wr, 4 op/s 2024-09-10T19:02:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:18 smithi107 ceph-mon[29404]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-09-10T19:02:18.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:18 smithi107 ceph-mon[40355]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-09-10T19:02:18.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:18 smithi162 ceph-mon[33228]: pgmap v595: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.8 KiB/s rd, 2.4 KiB/s wr, 5 op/s 2024-09-10T19:02:19.468 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:19] "GET /metrics HTTP/1.1" 200 228601 "" "Prometheus/2.43.0" 2024-09-10T19:02:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:20 smithi107 ceph-mon[29404]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T19:02:20.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:20 smithi107 ceph-mon[40355]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T19:02:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:20 smithi162 ceph-mon[33228]: pgmap v596: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 2.0 KiB/s wr, 4 op/s 2024-09-10T19:02:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:22 smithi107 ceph-mon[29404]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:02:22.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:22 smithi107 ceph-mon[40355]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:02:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:22 smithi162 ceph-mon[33228]: pgmap v597: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:02:22.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:23.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:24 smithi107 ceph-mon[29404]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 379 B/s wr, 1 op/s 2024-09-10T19:02:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:24 smithi107 ceph-mon[40355]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 379 B/s wr, 1 op/s 2024-09-10T19:02:24.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:24 smithi162 ceph-mon[33228]: pgmap v598: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 379 B/s wr, 1 op/s 2024-09-10T19:02:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:26 smithi107 ceph-mon[29404]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T19:02:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:26 smithi107 ceph-mon[40355]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T19:02:26.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:26 smithi162 ceph-mon[33228]: pgmap v599: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 938 B/s rd, 341 B/s wr, 1 op/s 2024-09-10T19:02:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:28 smithi107 ceph-mon[29404]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-10T19:02:28.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:28 smithi107 ceph-mon[40355]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-10T19:02:28.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:28 smithi162 ceph-mon[33228]: pgmap v600: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 597 B/s wr, 3 op/s 2024-09-10T19:02:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:29] "GET /metrics HTTP/1.1" 200 228601 "" "Prometheus/2.43.0" 2024-09-10T19:02:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:30 smithi107 ceph-mon[29404]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:30 smithi107 ceph-mon[40355]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:30.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:30 smithi162 ceph-mon[33228]: pgmap v601: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:31.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:31.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-mon[33228]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:32.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[29404]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[40355]: pgmap v602: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-71"}]: dispatch 2024-09-10T19:02:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:33.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:34 smithi107 ceph-mon[40355]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:34.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:34 smithi107 ceph-mon[40355]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T19:02:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:34 smithi107 ceph-mon[29404]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:34.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:34 smithi107 ceph-mon[29404]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T19:02:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:34 smithi162 ceph-mon[33228]: pgmap v603: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:02:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:34 smithi162 ceph-mon[33228]: osdmap e406: 8 total, 8 up, 8 in 2024-09-10T19:02:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:35 smithi162 ceph-mon[33228]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T19:02:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[29404]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T19:02:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:35.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[40355]: osdmap e407: 8 total, 8 up, 8 in 2024-09-10T19:02:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:35.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:36 smithi162 ceph-mon[33228]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:36 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:36 smithi162 ceph-mon[33228]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T19:02:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[29404]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[29404]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T19:02:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[40355]: pgmap v606: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-73","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[40355]: osdmap e408: 8 total, 8 up, 8 in 2024-09-10T19:02:36.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:37.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:37 smithi162 ceph-mon[33228]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T19:02:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:37 smithi107 ceph-mon[29404]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T19:02:37.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:37 smithi107 ceph-mon[40355]: osdmap e409: 8 total, 8 up, 8 in 2024-09-10T19:02:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:38 smithi162 ceph-mon[33228]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:38 smithi162 ceph-mon[33228]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T19:02:38.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:38 smithi107 ceph-mon[29404]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:38.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:38 smithi107 ceph-mon[29404]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T19:02:38.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:38 smithi107 ceph-mon[40355]: pgmap v609: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:38.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:38 smithi107 ceph-mon[40355]: osdmap e410: 8 total, 8 up, 8 in 2024-09-10T19:02:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:39] "GET /metrics HTTP/1.1" 200 228551 "" "Prometheus/2.43.0" 2024-09-10T19:02:40.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:40 smithi162 ceph-mon[33228]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T19:02:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:40 smithi107 ceph-mon[29404]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T19:02:40.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:40 smithi107 ceph-mon[40355]: pgmap v611: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.3 KiB/s wr, 2 op/s 2024-09-10T19:02:41.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:42.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-mon[33228]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-10T19:02:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:42.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:02:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[29404]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-10T19:02:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[40355]: pgmap v612: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 1023 B/s wr, 1 op/s 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:02:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:02:43.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:43.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:43.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:44.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:44 smithi162 ceph-mon[33228]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 929 B/s wr, 4 op/s 2024-09-10T19:02:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:44 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:44 smithi107 ceph-mon[40355]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 929 B/s wr, 4 op/s 2024-09-10T19:02:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:44 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:44 smithi107 ceph-mon[29404]: pgmap v613: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 929 B/s wr, 4 op/s 2024-09-10T19:02:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:44 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:46 smithi162 ceph-mon[33228]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 823 B/s wr, 3 op/s 2024-09-10T19:02:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:46 smithi107 ceph-mon[29404]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 823 B/s wr, 3 op/s 2024-09-10T19:02:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:46.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:46 smithi107 ceph-mon[40355]: pgmap v614: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 823 B/s wr, 3 op/s 2024-09-10T19:02:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:02:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:48 smithi162 ceph-mon[33228]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-10T19:02:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[29404]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-10T19:02:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[40355]: pgmap v615: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 102 B/s wr, 2 op/s 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-73"}]: dispatch 2024-09-10T19:02:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:49] "GET /metrics HTTP/1.1" 200 228557 "" "Prometheus/2.43.0" 2024-09-10T19:02:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:49 smithi162 ceph-mon[33228]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T19:02:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:49 smithi107 ceph-mon[29404]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T19:02:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:49 smithi107 ceph-mon[40355]: osdmap e411: 8 total, 8 up, 8 in 2024-09-10T19:02:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:50 smithi162 ceph-mon[33228]: pgmap v617: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:02:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:50 smithi162 ceph-mon[33228]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T19:02:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[29404]: pgmap v617: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:02:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[29404]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T19:02:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[40355]: pgmap v617: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:02:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[40355]: osdmap e412: 8 total, 8 up, 8 in 2024-09-10T19:02:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:51.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:02:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:02:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:51 smithi162 ceph-mon[33228]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T19:02:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[29404]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T19:02:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-75","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[40355]: osdmap e413: 8 total, 8 up, 8 in 2024-09-10T19:02:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-mon[33228]: pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-mon[33228]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T19:02:52.924 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:02:52.603+0000 7f1768522640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b7076ffc:test-rados-api-smithi107-80290-76::foo:28, removing 2024-09-10T19:02:52.925 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:02:52.603+0000 7f0065ea1640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b7076ffc:test-rados-api-smithi107-80290-76::foo:28, removing 2024-09-10T19:02:52.925 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:02:52.925 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:02:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:02:52.603+0000 7f6ac106d640 -1 snap_mapper.add_oid found existing snaps mapped on 64:b7076ffc:test-rados-api-smithi107-80290-76::foo:28, removing 2024-09-10T19:02:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:52 smithi107 ceph-mon[29404]: pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:52 smithi107 ceph-mon[29404]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T19:02:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:52 smithi107 ceph-mon[40355]: pgmap v620: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:02:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:52 smithi107 ceph-mon[40355]: osdmap e414: 8 total, 8 up, 8 in 2024-09-10T19:02:53.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T19:02:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T19:02:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: osdmap e415: 8 total, 8 up, 8 in 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-75"}]: dispatch 2024-09-10T19:02:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:54 smithi162 ceph-mon[33228]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:02:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:54 smithi162 ceph-mon[33228]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T19:02:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:54 smithi107 ceph-mon[40355]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:02:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:54 smithi107 ceph-mon[40355]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T19:02:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:54 smithi107 ceph-mon[29404]: pgmap v623: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:02:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:54 smithi107 ceph-mon[29404]: osdmap e416: 8 total, 8 up, 8 in 2024-09-10T19:02:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:55 smithi162 ceph-mon[33228]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T19:02:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:55.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[40355]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T19:02:55.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:55.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[29404]: osdmap e417: 8 total, 8 up, 8 in 2024-09-10T19:02:55.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:55.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:56.924 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:02:56.630+0000 7f6aca07f640 -1 snap_mapper.add_oid found existing snaps mapped on 64:a5434383:test-rados-api-smithi107-80290-78::foo:29, removing 2024-09-10T19:02:56.924 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:02:56.630+0000 7f1763518640 -1 snap_mapper.add_oid found existing snaps mapped on 64:a5434383:test-rados-api-smithi107-80290-78::foo:29, removing 2024-09-10T19:02:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-mon[33228]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:56.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:56.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-mon[33228]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T19:02:56.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[40355]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[40355]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T19:02:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[29404]: pgmap v626: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:02:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:02:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-77","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[29404]: osdmap e418: 8 total, 8 up, 8 in 2024-09-10T19:02:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:56.966 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:02:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T19:02:56.629+0000 7f7f6c966640 -1 snap_mapper.add_oid found existing snaps mapped on 64:a5434383:test-rados-api-smithi107-80290-78::foo:29, removing 2024-09-10T19:02:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:57 smithi162 ceph-mon[33228]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T19:02:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[29404]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T19:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[40355]: osdmap e419: 8 total, 8 up, 8 in 2024-09-10T19:02:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:02:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-77"}]: dispatch 2024-09-10T19:02:58.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:58 smithi162 ceph-mon[33228]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-10T19:02:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:58 smithi162 ceph-mon[33228]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T19:02:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:58 smithi107 ceph-mon[29404]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-10T19:02:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:58 smithi107 ceph-mon[29404]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T19:02:58.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:58 smithi107 ceph-mon[40355]: pgmap v629: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 2.5 KiB/s wr, 8 op/s 2024-09-10T19:02:58.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:58 smithi107 ceph-mon[40355]: osdmap e420: 8 total, 8 up, 8 in 2024-09-10T19:02:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:02:59] "GET /metrics HTTP/1.1" 200 228557 "" "Prometheus/2.43.0" 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T19:02:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:02:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T19:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: osdmap e421: 8 total, 8 up, 8 in 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-79","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: osdmap e422: 8 total, 8 up, 8 in 2024-09-10T19:02:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:02:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:00 smithi162 ceph-mon[33228]: pgmap v632: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T19:03:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:00 smithi107 ceph-mon[29404]: pgmap v632: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T19:03:00.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:00 smithi107 ceph-mon[40355]: pgmap v632: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 2.7 KiB/s wr, 5 op/s 2024-09-10T19:03:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:01 smithi162 ceph-mon[33228]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T19:03:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:01 smithi162 ceph-mon[33228]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T19:03:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[29404]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T19:03:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[29404]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T19:03:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[40355]: osdmap e423: 8 total, 8 up, 8 in 2024-09-10T19:03:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:01 smithi107 ceph-mon[40355]: osdmap e424: 8 total, 8 up, 8 in 2024-09-10T19:03:02.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-mon[33228]: pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:02.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:02 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[29404]: pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[40355]: pgmap v635: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg deep-scrub", "pgid": "223.c"}]: dispatch 2024-09-10T19:03:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:02 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:03 smithi162 ceph-mon[33228]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T19:03:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:03 smithi107 ceph-mon[29404]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T19:03:03.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:03 smithi107 ceph-mon[40355]: pgmap v637: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1.1 KiB/s wr, 3 op/s 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[29404]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[29404]: 223.c deep-scrub starts 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[29404]: 223.c deep-scrub ok 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[40355]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[40355]: 223.c deep-scrub starts 2024-09-10T19:03:06.669 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:06 smithi107 ceph-mon[40355]: 223.c deep-scrub ok 2024-09-10T19:03:06.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:06 smithi162 ceph-mon[33228]: pgmap v638: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:03:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:06 smithi162 ceph-mon[33228]: 223.c deep-scrub starts 2024-09-10T19:03:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:06 smithi162 ceph-mon[33228]: 223.c deep-scrub ok 2024-09-10T19:03:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:08 smithi162 ceph-mon[33228]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 673 B/s wr, 2 op/s 2024-09-10T19:03:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:08 smithi107 ceph-mon[29404]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 673 B/s wr, 2 op/s 2024-09-10T19:03:08.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:08 smithi107 ceph-mon[40355]: pgmap v639: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 673 B/s wr, 2 op/s 2024-09-10T19:03:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:09] "GET /metrics HTTP/1.1" 200 228555 "" "Prometheus/2.43.0" 2024-09-10T19:03:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:10 smithi162 ceph-mon[33228]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-10T19:03:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:10 smithi107 ceph-mon[29404]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-10T19:03:10.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:10 smithi107 ceph-mon[40355]: pgmap v640: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 596 B/s wr, 2 op/s 2024-09-10T19:03:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:12.554 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:12 smithi162 ceph-mon[33228]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:03:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:12 smithi107 ceph-mon[29404]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:03:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:12 smithi107 ceph-mon[40355]: pgmap v641: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:03:12.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:13.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:13.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:14 smithi162 ceph-mon[33228]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 442 B/s wr, 2 op/s 2024-09-10T19:03:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:14 smithi107 ceph-mon[29404]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 442 B/s wr, 2 op/s 2024-09-10T19:03:14.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:14 smithi107 ceph-mon[40355]: pgmap v642: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 442 B/s wr, 2 op/s 2024-09-10T19:03:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:16 smithi162 ceph-mon[33228]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:16 smithi107 ceph-mon[29404]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:16 smithi107 ceph-mon[40355]: pgmap v643: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:16.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:18 smithi162 ceph-mon[33228]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:18 smithi107 ceph-mon[29404]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:18 smithi107 ceph-mon[40355]: pgmap v644: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:19] "GET /metrics HTTP/1.1" 200 228544 "" "Prometheus/2.43.0" 2024-09-10T19:03:20.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:20 smithi162 ceph-mon[33228]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:20 smithi107 ceph-mon[29404]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:20.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:20 smithi107 ceph-mon[40355]: pgmap v645: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:21.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:21.965 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:03:21 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T19:03:21.677+0000 7f7f75978640 -1 snap_mapper.add_oid found existing snaps mapped on 223:349f0c25:test-rados-api-smithi107-80290-80::foo:2, removing 2024-09-10T19:03:22.174 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:03:21 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:03:21.677+0000 7f6ac186e640 -1 snap_mapper.add_oid found existing snaps mapped on 223:349f0c25:test-rados-api-smithi107-80290-80::foo:2, removing 2024-09-10T19:03:22.174 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:03:21 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:03:21.677+0000 7f00666a2640 -1 snap_mapper.add_oid found existing snaps mapped on 223:349f0c25:test-rados-api-smithi107-80290-80::foo:2, removing 2024-09-10T19:03:22.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-mon[33228]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[29404]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[40355]: pgmap v646: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-79"}]: dispatch 2024-09-10T19:03:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:23 smithi162 ceph-mon[33228]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T19:03:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:23 smithi107 ceph-mon[29404]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T19:03:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:23 smithi107 ceph-mon[40355]: osdmap e425: 8 total, 8 up, 8 in 2024-09-10T19:03:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:24 smithi162 ceph-mon[33228]: pgmap v648: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:24 smithi162 ceph-mon[33228]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T19:03:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[29404]: pgmap v648: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[29404]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T19:03:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[40355]: pgmap v648: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[40355]: osdmap e426: 8 total, 8 up, 8 in 2024-09-10T19:03:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:25.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:25 smithi162 ceph-mon[33228]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T19:03:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:25.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:25.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[29404]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T19:03:25.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-81","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[40355]: osdmap e427: 8 total, 8 up, 8 in 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:26 smithi162 ceph-mon[33228]: pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:26 smithi162 ceph-mon[33228]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T19:03:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[29404]: pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[29404]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T19:03:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[40355]: pgmap v651: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:03:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[40355]: osdmap e428: 8 total, 8 up, 8 in 2024-09-10T19:03:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:27 smithi162 ceph-mon[33228]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T19:03:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[29404]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T19:03:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[40355]: osdmap e429: 8 total, 8 up, 8 in 2024-09-10T19:03:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:28 smithi162 ceph-mon[33228]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:28 smithi162 ceph-mon[33228]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T19:03:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[29404]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[29404]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T19:03:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[40355]: pgmap v654: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[40355]: osdmap e430: 8 total, 8 up, 8 in 2024-09-10T19:03:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:29.368 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:29] "GET /metrics HTTP/1.1" 200 228544 "" "Prometheus/2.43.0" 2024-09-10T19:03:29.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:29 smithi162 ceph-mon[33228]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T19:03:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:29.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:29.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[40355]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T19:03:29.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:29.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[29404]: osdmap e431: 8 total, 8 up, 8 in 2024-09-10T19:03:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]: dispatch 2024-09-10T19:03:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-mon[33228]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T19:03:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-mon[33228]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T19:03:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[29404]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T19:03:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[29404]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T19:03:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[40355]: pgmap v657: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "512"}]': finished 2024-09-10T19:03:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[40355]: osdmap e432: 8 total, 8 up, 8 in 2024-09-10T19:03:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:30 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]: dispatch 2024-09-10T19:03:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T19:03:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:31 smithi162 ceph-mon[33228]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T19:03:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[29404]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "16384"}]': finished 2024-09-10T19:03:31.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[40355]: osdmap e433: 8 total, 8 up, 8 in 2024-09-10T19:03:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:32.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: pgmap v660: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-81","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: osdmap e434: 8 total, 8 up, 8 in 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-81"}]: dispatch 2024-09-10T19:03:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:33 smithi162 ceph-mon[33228]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T19:03:33.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:33 smithi107 ceph-mon[29404]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T19:03:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:33.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:33 smithi107 ceph-mon[40355]: osdmap e435: 8 total, 8 up, 8 in 2024-09-10T19:03:33.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:34 smithi162 ceph-mon[33228]: pgmap v663: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:34 smithi162 ceph-mon[33228]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T19:03:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[29404]: pgmap v663: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[29404]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T19:03:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[40355]: pgmap v663: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[40355]: osdmap e436: 8 total, 8 up, 8 in 2024-09-10T19:03:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:35 smithi162 ceph-mon[33228]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T19:03:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[29404]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T19:03:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-83","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[40355]: osdmap e437: 8 total, 8 up, 8 in 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[29404]: pgmap v666: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[29404]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T19:03:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[40355]: pgmap v666: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[40355]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:36 smithi162 ceph-mon[33228]: pgmap v666: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 117 KiB/s rd, 12 KiB/s wr, 218 op/s 2024-09-10T19:03:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:36 smithi162 ceph-mon[33228]: osdmap e438: 8 total, 8 up, 8 in 2024-09-10T19:03:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[29404]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T19:03:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[40355]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:37.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:37 smithi162 ceph-mon[33228]: osdmap e439: 8 total, 8 up, 8 in 2024-09-10T19:03:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[29404]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:38 smithi107 ceph-mon[40355]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: pgmap v669: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: osdmap e440: 8 total, 8 up, 8 in 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-83","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:38 smithi162 ceph-mon[33228]: osdmap e441: 8 total, 8 up, 8 in 2024-09-10T19:03:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:39] "GET /metrics HTTP/1.1" 200 228539 "" "Prometheus/2.43.0" 2024-09-10T19:03:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:40 smithi107 ceph-mon[29404]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:40 smithi107 ceph-mon[29404]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T19:03:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:40 smithi107 ceph-mon[40355]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:40 smithi107 ceph-mon[40355]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T19:03:40.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:40 smithi162 ceph-mon[33228]: pgmap v672: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:40.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:40 smithi162 ceph-mon[33228]: osdmap e442: 8 total, 8 up, 8 in 2024-09-10T19:03:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[29404]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T19:03:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[40355]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:41 smithi162 ceph-mon[33228]: osdmap e443: 8 total, 8 up, 8 in 2024-09-10T19:03:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-83"}]: dispatch 2024-09-10T19:03:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[29404]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[29404]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T19:03:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[40355]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[40355]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T19:03:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:42 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:42.825 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:42.825 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:42 smithi162 ceph-mon[33228]: pgmap v675: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:42.825 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:42 smithi162 ceph-mon[33228]: osdmap e444: 8 total, 8 up, 8 in 2024-09-10T19:03:42.825 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:42 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:03:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T19:03:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:03:43.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: osdmap e445: 8 total, 8 up, 8 in 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:03:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:44 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:45.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:45.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T19:03:45.009 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: pgmap v678: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-85","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: osdmap e446: 8 total, 8 up, 8 in 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:45.010 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:44 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:45.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:45 smithi162 ceph-mon[33228]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T19:03:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[29404]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T19:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[40355]: osdmap e447: 8 total, 8 up, 8 in 2024-09-10T19:03:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T19:03:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: pgmap v681: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.5 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_tier","val": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: osdmap e448: 8 total, 8 up, 8 in 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:03:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:47 smithi162 ceph-mon[33228]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T19:03:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[29404]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T19:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[40355]: osdmap e449: 8 total, 8 up, 8 in 2024-09-10T19:03:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:48 smithi162 ceph-mon[33228]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:48 smithi162 ceph-mon[33228]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T19:03:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[29404]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[29404]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T19:03:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[40355]: pgmap v684: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-85","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:48 smithi107 ceph-mon[40355]: osdmap e450: 8 total, 8 up, 8 in 2024-09-10T19:03:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:49] "GET /metrics HTTP/1.1" 200 228592 "" "Prometheus/2.43.0" 2024-09-10T19:03:49.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:49 smithi162 ceph-mon[33228]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T19:03:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:49 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:49 smithi107 ceph-mon[29404]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T19:03:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:49 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:49 smithi107 ceph-mon[40355]: osdmap e451: 8 total, 8 up, 8 in 2024-09-10T19:03:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:49 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: OK ] LibRadosTwoPoolsPP.ProxyRead (18093 ms) 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.CachePin 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.CachePin (22634 ms) 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.SetRedirectRead 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.SetRedirectRead (3014 ms) 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestPromoteRead 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestPromoteRead (3019 ms) 2024-09-10T19:03:50.531 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRefRead 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRefRead (3010 ms) 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestUnset 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestUnset (3012 ms) 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestDedupRefRead 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestDedupRefRead (4046 ms) 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount (39072 ms) 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapRefcount2 (17063 ms) 2024-09-10T19:03:50.532 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestTestSnapCreate 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestTestSnapCreate (4029 ms) 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRedirectAfterPromote (3018 ms) 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestCheckRefcountWhenModification (25096 ms) 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapIncCount 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapIncCount (15049 ms) 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvict 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.533 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvict (5055 ms) 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictPromote 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictPromote (4023 ms) 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrubs... 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapSizeMismatch (24696 ms) 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.DedupFlushRead 2024-09-10T19:03:50.534 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.DedupFlushRead (10080 ms) 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushSnap 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushSnap (9063 ms) 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestFlushDupCount 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:03:50.535 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestFlushDupCount (9073 ms) 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T19:03:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: pgmap v687: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: osdmap e452: 8 total, 8 up, 8 in 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-85"}]: dispatch 2024-09-10T19:03:51.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:03:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:03:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:51 smithi162 ceph-mon[33228]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T19:03:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:51 smithi107 ceph-mon[29404]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T19:03:51.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:51 smithi107 ceph-mon[40355]: osdmap e453: 8 total, 8 up, 8 in 2024-09-10T19:03:52.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:03:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:03:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:52 smithi162 ceph-mon[33228]: pgmap v690: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:52 smithi162 ceph-mon[33228]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T19:03:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[29404]: pgmap v690: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[29404]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T19:03:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[40355]: pgmap v690: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:03:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[40355]: osdmap e454: 8 total, 8 up, 8 in 2024-09-10T19:03:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:03:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:53 smithi162 ceph-mon[33228]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T19:03:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[29404]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T19:03:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-87","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:03:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[40355]: osdmap e455: 8 total, 8 up, 8 in 2024-09-10T19:03:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:03:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:03:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:54 smithi162 ceph-mon[33228]: pgmap v693: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:54 smithi162 ceph-mon[33228]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T19:03:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[40355]: pgmap v693: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:54.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[40355]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[29404]: pgmap v693: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[29404]: osdmap e456: 8 total, 8 up, 8 in 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:03:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:55 smithi162 ceph-mon[33228]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T19:03:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[29404]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T19:03:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:55.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:03:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[40355]: osdmap e457: 8 total, 8 up, 8 in 2024-09-10T19:03:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]: dispatch 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: pgmap v696: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]': finished 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: pgmap v696: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]': finished 2024-09-10T19:03:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: pgmap v696: 260 pgs: 32 unknown, 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_tier","val": "test-rados-api-smithi107-80290-89-test-flush"}]': finished 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: osdmap e458: 8 total, 8 up, 8 in 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:03:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:57 smithi162 ceph-mon[33228]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T19:03:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[29404]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[40355]: osdmap e459: 8 total, 8 up, 8 in 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:03:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:58 smithi162 ceph-mon[33228]: pgmap v699: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:58 smithi162 ceph-mon[33228]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T19:03:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[29404]: pgmap v699: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[29404]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T19:03:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[40355]: pgmap v699: 260 pgs: 260 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:03:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-87","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:03:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:58 smithi107 ceph-mon[40355]: osdmap e460: 8 total, 8 up, 8 in 2024-09-10T19:03:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:03:59] "GET /metrics HTTP/1.1" 200 228592 "" "Prometheus/2.43.0" 2024-09-10T19:03:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:59 smithi162 ceph-mon[33228]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T19:03:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:03:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:03:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:03:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[40355]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T19:03:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:03:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:03:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[29404]: osdmap e461: 8 total, 8 up, 8 in 2024-09-10T19:03:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:03:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:03:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:03:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-87"}]: dispatch 2024-09-10T19:04:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:00 smithi162 ceph-mon[33228]: pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:00 smithi162 ceph-mon[33228]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T19:04:00.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:00 smithi107 ceph-mon[40355]: pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:00.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:00 smithi107 ceph-mon[40355]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T19:04:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:00 smithi107 ceph-mon[29404]: pgmap v702: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:04:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:00 smithi107 ceph-mon[29404]: osdmap e462: 8 total, 8 up, 8 in 2024-09-10T19:04:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:01 smithi162 ceph-mon[33228]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T19:04:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[29404]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T19:04:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[40355]: osdmap e463: 8 total, 8 up, 8 in 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:02.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:04:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:02.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:02.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: pgmap v705: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-90","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: osdmap e464: 8 total, 8 up, 8 in 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:04:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:04:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:03 smithi162 ceph-mon[33228]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:04:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:04:03.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[29404]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:04:03.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:03.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-6","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:04:03.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:03 smithi107 ceph-mon[40355]: osdmap e465: 8 total, 8 up, 8 in 2024-09-10T19:04:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:04 smithi162 ceph-mon[33228]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:04 smithi162 ceph-mon[33228]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:04:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:04 smithi107 ceph-mon[29404]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:04 smithi107 ceph-mon[29404]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:04:04.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:04 smithi107 ceph-mon[40355]: pgmap v708: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:04 smithi107 ceph-mon[40355]: osdmap e466: 8 total, 8 up, 8 in 2024-09-10T19:04:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:05 smithi162 ceph-mon[33228]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:04:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[29404]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:04:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[40355]: osdmap e467: 8 total, 8 up, 8 in 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-90"}]: dispatch 2024-09-10T19:04:06.923 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:06 smithi162 ceph-mon[33228]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:06 smithi162 ceph-mon[33228]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:04:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:06 smithi107 ceph-mon[29404]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:06 smithi107 ceph-mon[29404]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:04:06.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:06 smithi107 ceph-mon[40355]: pgmap v711: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:04:06.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:06 smithi107 ceph-mon[40355]: osdmap e468: 8 total, 8 up, 8 in 2024-09-10T19:04:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:04:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: pgmap v714: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[40355]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: pgmap v714: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:07 smithi107 ceph-mon[29404]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: osdmap e469: 8 total, 8 up, 8 in 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: pgmap v714: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-92","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:07 smithi162 ceph-mon[33228]: osdmap e470: 8 total, 8 up, 8 in 2024-09-10T19:04:08.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:08.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:08.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[40355]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:04:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:08 smithi107 ceph-mon[29404]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:04:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:08 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:08 smithi162 ceph-mon[33228]: osdmap e471: 8 total, 8 up, 8 in 2024-09-10T19:04:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:09] "GET /metrics HTTP/1.1" 200 228595 "" "Prometheus/2.43.0" 2024-09-10T19:04:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:09 smithi107 ceph-mon[29404]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:09 smithi107 ceph-mon[40355]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:09 smithi162 ceph-mon[33228]: pgmap v717: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.5 KiB/s wr, 2 op/s 2024-09-10T19:04:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[29404]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:04:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[40355]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-mon[33228]: osdmap e472: 8 total, 8 up, 8 in 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-92"}]: dispatch 2024-09-10T19:04:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:11 smithi107 ceph-mon[29404]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:04:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:11 smithi107 ceph-mon[29404]: pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:11 smithi107 ceph-mon[40355]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:04:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:11 smithi107 ceph-mon[40355]: pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:12.075 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:11 smithi162 ceph-mon[33228]: osdmap e473: 8 total, 8 up, 8 in 2024-09-10T19:04:12.075 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:11 smithi162 ceph-mon[33228]: pgmap v720: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:12.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:12 smithi162 ceph-mon[33228]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:04:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:12.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:12 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:13.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[40355]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:04:13.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:13.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[29404]: osdmap e474: 8 total, 8 up, 8 in 2024-09-10T19:04:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:12 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:13 smithi162 ceph-mon[33228]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:04:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:13 smithi162 ceph-mon[33228]: pgmap v723: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[29404]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:04:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[29404]: pgmap v723: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-94","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[40355]: osdmap e475: 8 total, 8 up, 8 in 2024-09-10T19:04:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:13 smithi107 ceph-mon[40355]: pgmap v723: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:14 smithi162 ceph-mon[33228]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:04:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:14 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:14 smithi107 ceph-mon[29404]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:04:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:14 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:15.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:14 smithi107 ceph-mon[40355]: osdmap e476: 8 total, 8 up, 8 in 2024-09-10T19:04:15.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:14 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:04:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:15 smithi162 ceph-mon[33228]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:04:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:15 smithi162 ceph-mon[33228]: pgmap v726: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:15 smithi107 ceph-mon[29404]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:04:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:15 smithi107 ceph-mon[29404]: pgmap v726: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:15 smithi107 ceph-mon[40355]: osdmap e477: 8 total, 8 up, 8 in 2024-09-10T19:04:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:15 smithi107 ceph-mon[40355]: pgmap v726: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:04:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:17.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:17.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:17 smithi162 ceph-mon[33228]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T19:04:18.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:17 smithi107 ceph-mon[29404]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T19:04:18.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:17 smithi107 ceph-mon[40355]: pgmap v727: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.9 KiB/s rd, 2.0 KiB/s wr, 7 op/s 2024-09-10T19:04:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:19] "GET /metrics HTTP/1.1" 200 228628 "" "Prometheus/2.43.0" 2024-09-10T19:04:20.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:20 smithi162 ceph-mon[33228]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T19:04:20.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:20 smithi107 ceph-mon[29404]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T19:04:20.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:20 smithi107 ceph-mon[40355]: pgmap v728: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 1.2 KiB/s wr, 4 op/s 2024-09-10T19:04:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:22.609 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:22 smithi162 ceph-mon[33228]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T19:04:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:22 smithi107 ceph-mon[29404]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T19:04:22.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:22 smithi107 ceph-mon[40355]: pgmap v729: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1023 B/s wr, 3 op/s 2024-09-10T19:04:22.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:23.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:24 smithi162 ceph-mon[33228]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 861 B/s wr, 3 op/s 2024-09-10T19:04:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:24 smithi107 ceph-mon[29404]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 861 B/s wr, 3 op/s 2024-09-10T19:04:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:24 smithi107 ceph-mon[40355]: pgmap v730: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 861 B/s wr, 3 op/s 2024-09-10T19:04:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:26 smithi162 ceph-mon[33228]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 3 op/s 2024-09-10T19:04:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:26 smithi162 ceph-mon[33228]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:04:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:26 smithi107 ceph-mon[29404]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 3 op/s 2024-09-10T19:04:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:26 smithi107 ceph-mon[29404]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:04:26.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:26 smithi107 ceph-mon[40355]: pgmap v731: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 779 B/s wr, 3 op/s 2024-09-10T19:04:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:26 smithi107 ceph-mon[40355]: osdmap e478: 8 total, 8 up, 8 in 2024-09-10T19:04:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:28 smithi162 ceph-mon[33228]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:28 smithi107 ceph-mon[29404]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:28 smithi107 ceph-mon[40355]: pgmap v733: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:29] "GET /metrics HTTP/1.1" 200 228628 "" "Prometheus/2.43.0" 2024-09-10T19:04:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:30 smithi162 ceph-mon[33228]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:04:30.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:30 smithi162 ceph-mon[33228]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:30.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:30 smithi107 ceph-mon[40355]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:04:30.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:30 smithi107 ceph-mon[40355]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:30 smithi107 ceph-mon[29404]: osdmap e479: 8 total, 8 up, 8 in 2024-09-10T19:04:30.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:30 smithi107 ceph-mon[29404]: pgmap v735: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:31.152 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:31.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:31.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:31.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:32 smithi162 ceph-mon[33228]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:32 smithi107 ceph-mon[29404]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:32.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:32 smithi107 ceph-mon[40355]: pgmap v736: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 895 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:32.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:33.423 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:33.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:33.465 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:34.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:34 smithi162 ceph-mon[33228]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:34.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:34 smithi107 ceph-mon[29404]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:34.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:34 smithi107 ceph-mon[40355]: pgmap v737: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:36 smithi162 ceph-mon[33228]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[29404]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[40355]: pgmap v738: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-94"}]: dispatch 2024-09-10T19:04:37.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:37 smithi162 ceph-mon[33228]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:04:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:37 smithi107 ceph-mon[29404]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:04:37.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:37 smithi107 ceph-mon[40355]: osdmap e480: 8 total, 8 up, 8 in 2024-09-10T19:04:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:38 smithi162 ceph-mon[33228]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:38 smithi162 ceph-mon[33228]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:04:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[29404]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[29404]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[40355]: pgmap v740: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[40355]: osdmap e481: 8 total, 8 up, 8 in 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:39.328 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:39] "GET /metrics HTTP/1.1" 200 228595 "" "Prometheus/2.43.0" 2024-09-10T19:04:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:39 smithi162 ceph-mon[33228]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:04:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:39.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:39.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[40355]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:04:39.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-96","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[29404]: osdmap e482: 8 total, 8 up, 8 in 2024-09-10T19:04:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:04:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:40 smithi162 ceph-mon[33228]: pgmap v743: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:40 smithi162 ceph-mon[33228]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:04:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:40 smithi107 ceph-mon[40355]: pgmap v743: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:40 smithi107 ceph-mon[40355]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:04:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:40 smithi107 ceph-mon[29404]: pgmap v743: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:04:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:40 smithi107 ceph-mon[29404]: osdmap e483: 8 total, 8 up, 8 in 2024-09-10T19:04:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:42.626 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:42 smithi162 ceph-mon[33228]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-09-10T19:04:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:42 smithi107 ceph-mon[29404]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-09-10T19:04:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:42 smithi107 ceph-mon[40355]: pgmap v745: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 206 B/s wr, 1 op/s 2024-09-10T19:04:42.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:43.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:04:43.718 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:04:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:44 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-10T19:04:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:44 smithi162 ceph-mon[33228]: Adjusting osd_memory_target on smithi162 to 3248M 2024-09-10T19:04:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:44 smithi162 ceph-mon[33228]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:04:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-10T19:04:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[29404]: Adjusting osd_memory_target on smithi162 to 3248M 2024-09-10T19:04:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[29404]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:04:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi107 to 3247M 2024-09-10T19:04:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[40355]: Adjusting osd_memory_target on smithi162 to 3248M 2024-09-10T19:04:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:44 smithi107 ceph-mon[40355]: pgmap v746: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 853 B/s wr, 2 op/s 2024-09-10T19:04:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:46 smithi162 ceph-mon[33228]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 645 B/s rd, 645 B/s wr, 2 op/s 2024-09-10T19:04:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:46 smithi107 ceph-mon[29404]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 645 B/s rd, 645 B/s wr, 2 op/s 2024-09-10T19:04:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:46.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:46 smithi107 ceph-mon[40355]: pgmap v747: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 645 B/s rd, 645 B/s wr, 2 op/s 2024-09-10T19:04:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:04:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:48 smithi162 ceph-mon[33228]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 572 B/s wr, 2 op/s 2024-09-10T19:04:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:48 smithi107 ceph-mon[29404]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 572 B/s wr, 2 op/s 2024-09-10T19:04:48.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:48 smithi107 ceph-mon[40355]: pgmap v748: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 572 B/s wr, 2 op/s 2024-09-10T19:04:49.354 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:49] "GET /metrics HTTP/1.1" 200 228618 "" "Prometheus/2.43.0" 2024-09-10T19:04:49.674 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:04:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:04:49.354+0000 7f0066ea3640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c04d1e67:test-rados-api-smithi107-80290-97::foo:31, removing 2024-09-10T19:04:49.715 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:04:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T19:04:49.353+0000 7f7920f69640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c04d1e67:test-rados-api-smithi107-80290-97::foo:31, removing 2024-09-10T19:04:49.716 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:04:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T19:04:49.353+0000 7fc9cb0fd640 -1 snap_mapper.add_oid found existing snaps mapped on 64:c04d1e67:test-rados-api-smithi107-80290-97::foo:31, removing 2024-09-10T19:04:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-mon[33228]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:04:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[29404]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:04:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[40355]: pgmap v749: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:50.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-96"}]: dispatch 2024-09-10T19:04:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:04:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:04:51.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:51 smithi162 ceph-mon[33228]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:04:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:51 smithi107 ceph-mon[29404]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:04:51.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:51 smithi107 ceph-mon[40355]: osdmap e484: 8 total, 8 up, 8 in 2024-09-10T19:04:52.637 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:52 smithi162 ceph-mon[33228]: pgmap v751: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-10T19:04:52.637 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:52 smithi162 ceph-mon[33228]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:04:52.637 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.637 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[29404]: pgmap v751: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-10T19:04:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[29404]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:04:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[40355]: pgmap v751: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 204 B/s wr, 1 op/s 2024-09-10T19:04:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[40355]: osdmap e485: 8 total, 8 up, 8 in 2024-09-10T19:04:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:04:52.923 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:04:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:04:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:53 smithi162 ceph-mon[33228]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:04:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[29404]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:04:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-98","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:04:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[40355]: osdmap e486: 8 total, 8 up, 8 in 2024-09-10T19:04:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:04:53.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:04:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:54 smithi162 ceph-mon[33228]: pgmap v754: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:04:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:04:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:54 smithi162 ceph-mon[33228]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:04:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[29404]: pgmap v754: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:04:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:04:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[29404]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[40355]: pgmap v754: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[40355]: osdmap e487: 8 total, 8 up, 8 in 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]: dispatch 2024-09-10T19:04:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:55 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:04:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]': finished 2024-09-10T19:04:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:55 smithi162 ceph-mon[33228]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:04:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:04:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]': finished 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[29404]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-98", "mode": "writeback"}]': finished 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[40355]: osdmap e488: 8 total, 8 up, 8 in 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:04:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:56 smithi162 ceph-mon[33228]: pgmap v757: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:04:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:56 smithi162 ceph-mon[33228]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:04:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[40355]: pgmap v757: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:04:56.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[40355]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[29404]: pgmap v757: 228 pgs: 32 creating+peering, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[29404]: osdmap e489: 8 total, 8 up, 8 in 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:04:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:57 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:04:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:04:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:57 smithi162 ceph-mon[33228]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:04:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:57.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[29404]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[40355]: osdmap e490: 8 total, 8 up, 8 in 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]: dispatch 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:04:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:58 smithi162 ceph-mon[33228]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:04:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T19:04:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:04:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[29404]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: pgmap v760: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_count","val": "1"}]': finished 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: osdmap e491: 8 total, 8 up, 8 in 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:04:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:58 smithi107 ceph-mon[40355]: osdmap e492: 8 total, 8 up, 8 in 2024-09-10T19:04:59.431 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:04:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:04:59] "GET /metrics HTTP/1.1" 200 228618 "" "Prometheus/2.43.0" 2024-09-10T19:04:59.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:04:59.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:04:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:04:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:04:59.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:04:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:04:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:04:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:04:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]: dispatch 2024-09-10T19:05:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[29404]: pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:05:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[29404]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[40355]: pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[40355]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-mon[33228]: pgmap v763: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:05:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-98","var": "target_max_objects","val": "250"}]': finished 2024-09-10T19:05:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-mon[33228]: osdmap e493: 8 total, 8 up, 8 in 2024-09-10T19:05:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[29404]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[40355]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:01 smithi162 ceph-mon[33228]: osdmap e494: 8 total, 8 up, 8 in 2024-09-10T19:05:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]: dispatch 2024-09-10T19:05:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:02.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[40355]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:02.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:05:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[40355]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:05:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[29404]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:05:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:02 smithi107 ceph-mon[29404]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:05:02.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:02 smithi162 ceph-mon[33228]: pgmap v766: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-98"}]': finished 2024-09-10T19:05:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:02 smithi162 ceph-mon[33228]: osdmap e495: 8 total, 8 up, 8 in 2024-09-10T19:05:03.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:03 smithi107 ceph-mon[40355]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:05:03.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:03 smithi107 ceph-mon[29404]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:05:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:03 smithi162 ceph-mon[33228]: osdmap e496: 8 total, 8 up, 8 in 2024-09-10T19:05:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:03.924 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:05:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T19:05:03.491753196Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:05:03.924 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:05:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=cleanup t=2024-09-10T19:05:03.493053067Z level=info msg="Completed cleanup jobs" duration=12.574193ms 2024-09-10T19:05:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:04 smithi162 ceph-mon[33228]: pgmap v769: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:04 smithi162 ceph-mon[33228]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T19:05:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[29404]: pgmap v769: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[29404]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T19:05:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[40355]: pgmap v769: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[40355]: osdmap e497: 8 total, 8 up, 8 in 2024-09-10T19:05:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:05 smithi162 ceph-mon[33228]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T19:05:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:05.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[29404]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T19:05:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:05.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-100","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[40355]: osdmap e498: 8 total, 8 up, 8 in 2024-09-10T19:05:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:05.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[29404]: pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[29404]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[40355]: pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:05:06.867 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[40355]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T19:05:06.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:06.868 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:06 smithi162 ceph-mon[33228]: pgmap v772: 228 pgs: 32 unknown, 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:05:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:06 smithi162 ceph-mon[33228]: osdmap e499: 8 total, 8 up, 8 in 2024-09-10T19:05:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:07 smithi162 ceph-mon[33228]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T19:05:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[29404]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T19:05:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:07.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-6", "overlaypool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[40355]: osdmap e500: 8 total, 8 up, 8 in 2024-09-10T19:05:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:07.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]: dispatch 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]': finished 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]': finished 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: pgmap v775: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-100", "mode": "writeback"}]': finished 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: osdmap e501: 8 total, 8 up, 8 in 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:09] "GET /metrics HTTP/1.1" 200 228571 "" "Prometheus/2.43.0" 2024-09-10T19:05:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:09 smithi162 ceph-mon[33228]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T19:05:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[40355]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T19:05:09.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[29404]: osdmap e502: 8 total, 8 up, 8 in 2024-09-10T19:05:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:10.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-mon[33228]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:10.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:10.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-mon[33228]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T19:05:10.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:10.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[29404]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[29404]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[40355]: pgmap v778: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[40355]: osdmap e503: 8 total, 8 up, 8 in 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:11.173 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T19:05:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: osdmap e504: 8 total, 8 up, 8 in 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]: dispatch 2024-09-10T19:05:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-100","var": "min_read_recency_for_promote","val": "10000"}]': finished 2024-09-10T19:05:12.826 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:12 smithi162 ceph-mon[33228]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:12 smithi162 ceph-mon[33228]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T19:05:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[29404]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[29404]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T19:05:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[40355]: pgmap v781: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[40355]: osdmap e505: 8 total, 8 up, 8 in 2024-09-10T19:05:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:13 smithi162 ceph-mon[33228]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T19:05:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[29404]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T19:05:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]': finished 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[40355]: osdmap e506: 8 total, 8 up, 8 in 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]: dispatch 2024-09-10T19:05:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:14 smithi162 ceph-mon[33228]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:05:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:14 smithi162 ceph-mon[33228]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T19:05:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[29404]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:05:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:15.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[29404]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T19:05:15.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[40355]: pgmap v784: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:05:15.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-100"}]': finished 2024-09-10T19:05:15.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:14 smithi107 ceph-mon[40355]: osdmap e507: 8 total, 8 up, 8 in 2024-09-10T19:05:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:15 smithi162 ceph-mon[33228]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T19:05:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:15 smithi162 ceph-mon[33228]: pgmap v787: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:15 smithi162 ceph-mon[33228]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T19:05:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:16.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[40355]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T19:05:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[40355]: pgmap v787: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[40355]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T19:05:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:16.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[29404]: osdmap e508: 8 total, 8 up, 8 in 2024-09-10T19:05:16.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[29404]: pgmap v787: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:16.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[29404]: osdmap e509: 8 total, 8 up, 8 in 2024-09-10T19:05:16.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:16.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:17.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:17.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-102","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: osdmap e510: 8 total, 8 up, 8 in 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd dump","format":"json"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]: dispatch 2024-09-10T19:05:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:18 smithi162 ceph-mon[33228]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:05:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:18 smithi162 ceph-mon[33228]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T19:05:18.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:18.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:18.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[40355]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:18.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[40355]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[29404]: pgmap v790: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "fingerprint_algorithm","val": "sha1"}]': finished 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[29404]: osdmap e511: 8 total, 8 up, 8 in 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]: dispatch 2024-09-10T19:05:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:19] "GET /metrics HTTP/1.1" 200 228586 "" "Prometheus/2.43.0" 2024-09-10T19:05:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:05:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:19 smithi162 ceph-mon[33228]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T19:05:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:19.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[29404]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_chunk_algorithm","val": "fastcdc"}]': finished 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[40355]: osdmap e512: 8 total, 8 up, 8 in 2024-09-10T19:05:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:19.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]: dispatch 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: pgmap v793: 228 pgs: 228 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-102","var": "dedup_cdc_chunk_size","val": "1024"}]': finished 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: osdmap e513: 8 total, 8 up, 8 in 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-6"}]: dispatch 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/909364778' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:20.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-6", "tierpool": "test-rados-api-smithi107-80290-102"}]: dispatch 2024-09-10T19:05:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:21 smithi162 ceph-mon[33228]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T19:05:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:21 smithi107 ceph-mon[29404]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T19:05:21.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:21 smithi107 ceph-mon[40355]: osdmap e514: 8 total, 8 up, 8 in 2024-09-10T19:05:22.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-mon[33228]: pgmap v796: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:22.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-mon[33228]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T19:05:22.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[29404]: pgmap v796: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[29404]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[40355]: pgmap v796: 196 pgs: 196 active+clean; 8.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[40355]: osdmap e515: 8 total, 8 up, 8 in 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:23 smithi162 ceph-mon[33228]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T19:05:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[29404]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T19:05:23.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[40355]: osdmap e516: 8 total, 8 up, 8 in 2024-09-10T19:05:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:24 smithi162 ceph-mon[33228]: pgmap v799: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:24 smithi162 ceph-mon[33228]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T19:05:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:24 smithi107 ceph-mon[29404]: pgmap v799: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:24 smithi107 ceph-mon[29404]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T19:05:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:24 smithi107 ceph-mon[40355]: pgmap v799: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:24 smithi107 ceph-mon[40355]: osdmap e517: 8 total, 8 up, 8 in 2024-09-10T19:05:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:25 smithi162 ceph-mon[33228]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T19:05:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:25.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:25 smithi107 ceph-mon[29404]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T19:05:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "LibRadosTierECPP_smithi107-80290-104", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:25 smithi107 ceph-mon[40355]: osdmap e518: 8 total, 8 up, 8 in 2024-09-10T19:05:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:26 smithi162 ceph-mon[33228]: pgmap v802: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:26 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:26 smithi162 ceph-mon[33228]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T19:05:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[29404]: pgmap v802: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[29404]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T19:05:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[40355]: pgmap v802: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:26 smithi107 ceph-mon[40355]: osdmap e519: 8 total, 8 up, 8 in 2024-09-10T19:05:27.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:27 smithi162 ceph-mon[33228]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T19:05:27.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:27.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:27 smithi107 ceph-mon[29404]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T19:05:27.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:27.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:27 smithi107 ceph-mon[40355]: osdmap e520: 8 total, 8 up, 8 in 2024-09-10T19:05:27.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:28 smithi162 ceph-mon[33228]: pgmap v805: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:28 smithi162 ceph-mon[33228]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T19:05:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[29404]: pgmap v805: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[29404]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T19:05:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[40355]: pgmap v805: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:28 smithi107 ceph-mon[40355]: osdmap e521: 8 total, 8 up, 8 in 2024-09-10T19:05:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:29] "GET /metrics HTTP/1.1" 200 228586 "" "Prometheus/2.43.0" 2024-09-10T19:05:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:29 smithi162 ceph-mon[33228]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T19:05:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:29 smithi162 ceph-mon[33228]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T19:05:29.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:29.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[29404]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T19:05:29.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:29.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[29404]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[40355]: osdmap e522: 8 total, 8 up, 8 in 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-107-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[40355]: osdmap e523: 8 total, 8 up, 8 in 2024-09-10T19:05:29.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:30.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:30.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:30 smithi162 ceph-mon[33228]: pgmap v808: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:30.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:30.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:30 smithi162 ceph-mon[33228]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T19:05:30.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:30 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[29404]: pgmap v808: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[29404]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T19:05:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[40355]: pgmap v808: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:05:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[40355]: osdmap e524: 8 total, 8 up, 8 in 2024-09-10T19:05:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:30 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:31 smithi162 ceph-mon[33228]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T19:05:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[29404]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T19:05:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:31.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:31.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-107", "overlaypool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:31.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[40355]: osdmap e525: 8 total, 8 up, 8 in 2024-09-10T19:05:31.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:32.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: pgmap v811: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T19:05:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]: dispatch 2024-09-10T19:05:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: pgmap v811: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]: dispatch 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: pgmap v811: 236 pgs: 32 unknown, 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-107-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: osdmap e526: 8 total, 8 up, 8 in 2024-09-10T19:05:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]: dispatch 2024-09-10T19:05:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:33 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]': finished 2024-09-10T19:05:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:33 smithi162 ceph-mon[33228]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T19:05:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:33 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]': finished 2024-09-10T19:05:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[29404]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T19:05:33.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-107"}]': finished 2024-09-10T19:05:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[40355]: osdmap e527: 8 total, 8 up, 8 in 2024-09-10T19:05:33.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:33 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]: dispatch 2024-09-10T19:05:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:34 smithi162 ceph-mon[33228]: pgmap v814: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:34 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:34 smithi162 ceph-mon[33228]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T19:05:34.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[40355]: pgmap v814: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[40355]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[29404]: pgmap v814: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2327647482' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-107", "tierpool": "test-rados-api-smithi107-80290-107-cache"}]': finished 2024-09-10T19:05:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:34 smithi107 ceph-mon[29404]: osdmap e528: 8 total, 8 up, 8 in 2024-09-10T19:05:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:36 smithi162 ceph-mon[33228]: pgmap v817: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:36 smithi162 ceph-mon[33228]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T19:05:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:36 smithi107 ceph-mon[29404]: pgmap v817: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:36 smithi107 ceph-mon[29404]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T19:05:36.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:36 smithi107 ceph-mon[40355]: pgmap v817: 236 pgs: 236 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:36.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:36 smithi107 ceph-mon[40355]: osdmap e529: 8 total, 8 up, 8 in 2024-09-10T19:05:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:37 smithi162 ceph-mon[33228]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T19:05:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[29404]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T19:05:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[40355]: osdmap e530: 8 total, 8 up, 8 in 2024-09-10T19:05:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:37.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:38 smithi162 ceph-mon[33228]: pgmap v820: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:38 smithi162 ceph-mon[33228]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T19:05:39.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:39.215 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:39] "GET /metrics HTTP/1.1" 200 231062 "" "Prometheus/2.43.0" 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[29404]: pgmap v820: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[29404]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[40355]: pgmap v820: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-109", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[40355]: osdmap e531: 8 total, 8 up, 8 in 2024-09-10T19:05:39.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:40.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:39 smithi162 ceph-mon[33228]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T19:05:40.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:40.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:39 smithi162 ceph-mon[33228]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T19:05:40.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[29404]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T19:05:40.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:40.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[29404]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T19:05:40.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[40355]: osdmap e532: 8 total, 8 up, 8 in 2024-09-10T19:05:40.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-109", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:40.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:39 smithi107 ceph-mon[40355]: osdmap e533: 8 total, 8 up, 8 in 2024-09-10T19:05:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:41.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:40 smithi162 ceph-mon[33228]: pgmap v823: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:41.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:40 smithi162 ceph-mon[33228]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T19:05:41.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:41.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[29404]: pgmap v823: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:41.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[29404]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T19:05:41.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[40355]: pgmap v823: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[40355]: osdmap e534: 8 total, 8 up, 8 in 2024-09-10T19:05:41.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:05:42.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-mon[33228]: pgmap v826: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-mon[33228]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T19:05:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:42 smithi162 ceph-mon[33228]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[29404]: pgmap v826: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[29404]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[29404]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[40355]: pgmap v826: 212 pgs: 40 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-109-cache","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[40355]: osdmap e535: 8 total, 8 up, 8 in 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:42 smithi107 ceph-mon[40355]: Health check update: 4 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T19:05:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:43.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T19:05:43.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:43.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T19:05:43.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: osdmap e536: 8 total, 8 up, 8 in 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-109", "overlaypool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: osdmap e537: 8 total, 8 up, 8 in 2024-09-10T19:05:44.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]: dispatch 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: pgmap v829: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T19:05:44.914 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: pgmap v829: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: pgmap v829: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-109-cache", "mode": "writeback"}]': finished 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: osdmap e538: 8 total, 8 up, 8 in 2024-09-10T19:05:44.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: pgmap v832: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:46.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: pgmap v832: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:46.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: pgmap v832: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: osdmap e539: 8 total, 8 up, 8 in 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:05:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:46.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:05:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T19:05:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[29404]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:47.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:47.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:47 smithi107 ceph-mon[40355]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: osdmap e540: 8 total, 8 up, 8 in 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:05:48.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:47 smithi162 ceph-mon[33228]: osdmap e541: 8 total, 8 up, 8 in 2024-09-10T19:05:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: pgmap v835: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[29404]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: pgmap v835: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:48.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T19:05:48.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:48 smithi107 ceph-mon[40355]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: pgmap v835: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]: dispatch 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-109-cache","var": "min_read_recency_for_promote","val": "4"}]': finished 2024-09-10T19:05:49.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:48 smithi162 ceph-mon[33228]: osdmap e542: 8 total, 8 up, 8 in 2024-09-10T19:05:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:49] "GET /metrics HTTP/1.1" 200 231029 "" "Prometheus/2.43.0" 2024-09-10T19:05:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:49 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:49 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:50.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:49 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[29404]: pgmap v838: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[29404]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[40355]: pgmap v838: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[40355]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T19:05:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:05:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:05:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:50 smithi162 ceph-mon[33228]: pgmap v838: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:50 smithi162 ceph-mon[33228]: osdmap e543: 8 total, 8 up, 8 in 2024-09-10T19:05:51.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]: dispatch 2024-09-10T19:05:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[29404]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T19:05:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[29404]: pgmap v841: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[40355]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T19:05:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:51 smithi107 ceph-mon[40355]: pgmap v841: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:52.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-109", "tierpool": "test-rados-api-smithi107-80290-109-cache"}]': finished 2024-09-10T19:05:52.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:51 smithi162 ceph-mon[33228]: osdmap e544: 8 total, 8 up, 8 in 2024-09-10T19:05:52.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:51 smithi162 ceph-mon[33228]: pgmap v841: 212 pgs: 212 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:05:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:52 smithi107 ceph-mon[29404]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T19:05:53.015 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:52 smithi107 ceph-mon[40355]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T19:05:53.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:52 smithi162 ceph-mon[33228]: osdmap e545: 8 total, 8 up, 8 in 2024-09-10T19:05:53.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:05:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:05:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T19:05:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:53.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:53.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T19:05:53.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: osdmap e546: 8 total, 8 up, 8 in 2024-09-10T19:05:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:54.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:05:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: pgmap v844: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: osdmap e547: 8 total, 8 up, 8 in 2024-09-10T19:05:54.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]: dispatch 2024-09-10T19:05:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[29404]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T19:05:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:54 smithi107 ceph-mon[40355]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T19:05:55.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:54 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:05:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1029624146' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-109"}]': finished 2024-09-10T19:05:55.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:54 smithi162 ceph-mon[33228]: osdmap e548: 8 total, 8 up, 8 in 2024-09-10T19:05:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:55 smithi162 ceph-mon[33228]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:55 smithi162 ceph-mon[33228]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T19:05:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:56.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[29404]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:56.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[29404]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T19:05:56.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:56.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[40355]: pgmap v847: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:05:56.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[40355]: osdmap e549: 8 total, 8 up, 8 in 2024-09-10T19:05:56.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:57.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:57.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[40355]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T19:05:57.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[40355]: pgmap v850: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:05:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[29404]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T19:05:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:57 smithi107 ceph-mon[29404]: pgmap v850: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:05:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:57 smithi162 ceph-mon[33228]: osdmap e550: 8 total, 8 up, 8 in 2024-09-10T19:05:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]: dispatch 2024-09-10T19:05:58.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:57 smithi162 ceph-mon[33228]: pgmap v850: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:05:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:58.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1249656655' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"LibRadosTierECPP_smithi107-80290-104"}]': finished 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: osdmap e551: 8 total, 8 up, 8 in 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:59.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]: dispatch 2024-09-10T19:05:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:05:59] "GET /metrics HTTP/1.1" 200 231029 "" "Prometheus/2.43.0" 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:00.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:05:59 smithi162 ceph-mon[33228]: pgmap v853: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:06:00.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:06:00.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[29404]: pgmap v853: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "testprofile-test-rados-api-smithi107-80290-111", "profile": [ "k=2", "m=1", "crush-failure-domain=osd"]}]': finished 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: osdmap e552: 8 total, 8 up, 8 in 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:00.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:00.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:05:59 smithi107 ceph-mon[40355]: pgmap v853: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:06:01.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:00 smithi162 ceph-mon[33228]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T19:06:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:01.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:00 smithi107 ceph-mon[29404]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T19:06:01.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:00 smithi107 ceph-mon[40355]: osdmap e553: 8 total, 8 up, 8 in 2024-09-10T19:06:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:01 smithi162 ceph-mon[33228]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T19:06:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:02.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:01 smithi162 ceph-mon[33228]: pgmap v856: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:02.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:02.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[29404]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T19:06:02.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[29404]: pgmap v856: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "test-rados-api-smithi107-80290-111", "pool_type":"erasure", "pg_num":8, "pgp_num":8, "erasure_code_profile":"testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[40355]: osdmap e554: 8 total, 8 up, 8 in 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:02.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:01 smithi107 ceph-mon[40355]: pgmap v856: 172 pgs: 8 unknown, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:03.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:02 smithi162 ceph-mon[33228]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T19:06:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:03.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:03.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[40355]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T19:06:03.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:03.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:03.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[29404]: osdmap e555: 8 total, 8 up, 8 in 2024-09-10T19:06:03.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:03.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:03 smithi162 ceph-mon[33228]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:04.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:04.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:04.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[29404]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-112","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: osdmap e556: 8 total, 8 up, 8 in 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:04.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:03 smithi107 ceph-mon[40355]: pgmap v859: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:04 smithi162 ceph-mon[33228]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T19:06:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:05.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:05.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[40355]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T19:06:05.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:05.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:05.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[29404]: osdmap e557: 8 total, 8 up, 8 in 2024-09-10T19:06:05.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:05.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:06.116 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:06.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[29404]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T19:06:06.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.117 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[29404]: pgmap v862: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:06.117 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:06.118 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[40355]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T19:06:06.118 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.118 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.118 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:05 smithi107 ceph-mon[40355]: pgmap v862: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:05 smithi162 ceph-mon[33228]: osdmap e558: 8 total, 8 up, 8 in 2024-09-10T19:06:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:05 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:06.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:05 smithi162 ceph-mon[33228]: pgmap v862: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:06 smithi162 ceph-mon[33228]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T19:06:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:06 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:07.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:06 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:07.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:07.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[29404]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T19:06:07.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:07.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:07.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:07.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[40355]: osdmap e559: 8 total, 8 up, 8 in 2024-09-10T19:06:07.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:07.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:06 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]: dispatch 2024-09-10T19:06:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:07 smithi162 ceph-mon[33228]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T19:06:08.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:07 smithi162 ceph-mon[33228]: pgmap v865: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[29404]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[29404]: pgmap v865: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-112"}]': finished 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[40355]: osdmap e560: 8 total, 8 up, 8 in 2024-09-10T19:06:08.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:07 smithi107 ceph-mon[40355]: pgmap v865: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:06:09.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:08 smithi107 ceph-mon[29404]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T19:06:09.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:08 smithi107 ceph-mon[40355]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T19:06:09.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:08 smithi162 ceph-mon[33228]: osdmap e561: 8 total, 8 up, 8 in 2024-09-10T19:06:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:09] "GET /metrics HTTP/1.1" 200 226174 "" "Prometheus/2.43.0" 2024-09-10T19:06:09.799 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[29404]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T19:06:09.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:09.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:09.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:09.800 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[29404]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:09 smithi162 ceph-mon[33228]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T19:06:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:09 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:10.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:09 smithi162 ceph-mon[33228]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[40355]: osdmap e562: 8 total, 8 up, 8 in 2024-09-10T19:06:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:10.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:10.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:09 smithi107 ceph-mon[40355]: pgmap v868: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:10 smithi162 ceph-mon[33228]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T19:06:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:11.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:11.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[29404]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-114","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[40355]: osdmap e563: 8 total, 8 up, 8 in 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:11.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:12.075 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:12.076 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:11 smithi162 ceph-mon[33228]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T19:06:12.076 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.076 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.076 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:11 smithi162 ceph-mon[33228]: pgmap v871: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:12.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[29404]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T19:06:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[29404]: pgmap v871: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[40355]: osdmap e564: 8 total, 8 up, 8 in 2024-09-10T19:06:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:12.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:11 smithi107 ceph-mon[40355]: pgmap v871: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:13.086 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:13.086 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:13.086 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:12 smithi162 ceph-mon[33228]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T19:06:13.087 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:13.087 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:13.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[29404]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[40355]: osdmap e565: 8 total, 8 up, 8 in 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:13.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]: dispatch 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]': finished 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:13 smithi162 ceph-mon[33228]: pgmap v874: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:14.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]': finished 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[29404]: pgmap v874: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:14.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-114", "mode": "writeback"}]': finished 2024-09-10T19:06:14.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: osdmap e566: 8 total, 8 up, 8 in 2024-09-10T19:06:14.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:14.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:13 smithi107 ceph-mon[40355]: pgmap v874: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:14 smithi162 ceph-mon[33228]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T19:06:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:14 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[29404]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[40355]: osdmap e567: 8 total, 8 up, 8 in 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]: dispatch 2024-09-10T19:06:15.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:14 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:15 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:15 smithi162 ceph-mon[33228]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T19:06:16.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:15 smithi162 ceph-mon[33228]: pgmap v877: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[29404]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T19:06:16.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[29404]: pgmap v877: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-114"}]': finished 2024-09-10T19:06:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[40355]: osdmap e568: 8 total, 8 up, 8 in 2024-09-10T19:06:16.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:15 smithi107 ceph-mon[40355]: pgmap v877: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:06:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:16 smithi162 ceph-mon[33228]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T19:06:17.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:17.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:16 smithi107 ceph-mon[40355]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T19:06:17.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:16 smithi107 ceph-mon[29404]: osdmap e569: 8 total, 8 up, 8 in 2024-09-10T19:06:17.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: pgmap v880: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:18.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:17 smithi162 ceph-mon[33228]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T19:06:18.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T19:06:18.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: pgmap v880: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:18.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:18.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[29404]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T19:06:18.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: osdmap e570: 8 total, 8 up, 8 in 2024-09-10T19:06:18.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:18.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: pgmap v880: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:18.217 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-116","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:18.218 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:17 smithi107 ceph-mon[40355]: osdmap e571: 8 total, 8 up, 8 in 2024-09-10T19:06:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:19] "GET /metrics HTTP/1.1" 200 226232 "" "Prometheus/2.43.0" 2024-09-10T19:06:20.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:19 smithi162 ceph-mon[33228]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T19:06:20.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:19 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:20.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:19 smithi162 ceph-mon[33228]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:20.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[29404]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T19:06:20.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[29404]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[40355]: osdmap e572: 8 total, 8 up, 8 in 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:20.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:19 smithi107 ceph-mon[40355]: pgmap v883: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:21.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:20 smithi162 ceph-mon[33228]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T19:06:21.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:21.175 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:21.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:21.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[40355]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[29404]: osdmap e573: 8 total, 8 up, 8 in 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:21.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:22.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:21 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:22.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T19:06:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:22.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: osdmap e574: 8 total, 8 up, 8 in 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]: dispatch 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: pgmap v886: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:22.217 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:21 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:23.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:23.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]': finished 2024-09-10T19:06:23.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:22 smithi162 ceph-mon[33228]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T19:06:23.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:23.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:22 smithi162 ceph-mon[33228]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:23.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]': finished 2024-09-10T19:06:23.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[29404]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T19:06:23.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:23.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[29404]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-116", "mode": "writeback"}]': finished 2024-09-10T19:06:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[40355]: osdmap e575: 8 total, 8 up, 8 in 2024-09-10T19:06:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:23.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:22 smithi107 ceph-mon[40355]: from='mon.? -' entity='mon.' cmd=[{"prefix": "pg scrub", "pgid": "244.7"}]: dispatch 2024-09-10T19:06:24.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:23 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:24.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:23 smithi162 ceph-mon[33228]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:06:24.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:23 smithi162 ceph-mon[33228]: 244.7 scrub starts 2024-09-10T19:06:24.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:23 smithi162 ceph-mon[33228]: 244.7 scrub ok 2024-09-10T19:06:24.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:24.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[29404]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:06:24.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[29404]: 244.7 scrub starts 2024-09-10T19:06:24.216 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[29404]: 244.7 scrub ok 2024-09-10T19:06:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[40355]: pgmap v888: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 1.6 KiB/s wr, 3 op/s 2024-09-10T19:06:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[40355]: 244.7 scrub starts 2024-09-10T19:06:24.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:23 smithi107 ceph-mon[40355]: 244.7 scrub ok 2024-09-10T19:06:25.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:24 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:25.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:24 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:25.215 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:24 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:26.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:25 smithi162 ceph-mon[33228]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:06:26.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:25 smithi107 ceph-mon[29404]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:06:26.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:25 smithi107 ceph-mon[40355]: pgmap v889: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:06:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:28 smithi162 ceph-mon[33228]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 965 B/s wr, 2 op/s 2024-09-10T19:06:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:28 smithi107 ceph-mon[29404]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 965 B/s wr, 2 op/s 2024-09-10T19:06:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:28 smithi107 ceph-mon[40355]: pgmap v890: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 965 B/s wr, 2 op/s 2024-09-10T19:06:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:29] "GET /metrics HTTP/1.1" 200 226232 "" "Prometheus/2.43.0" 2024-09-10T19:06:30.673 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:30 smithi162 ceph-mon[33228]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 851 B/s wr, 2 op/s 2024-09-10T19:06:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:30 smithi107 ceph-mon[29404]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 851 B/s wr, 2 op/s 2024-09-10T19:06:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:30 smithi107 ceph-mon[40355]: pgmap v891: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 851 B/s wr, 2 op/s 2024-09-10T19:06:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:32 smithi162 ceph-mon[33228]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T19:06:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:32.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[29404]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T19:06:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:32 smithi107 ceph-mon[40355]: pgmap v892: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 716 B/s wr, 2 op/s 2024-09-10T19:06:33.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:34 smithi162 ceph-mon[33228]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 627 B/s wr, 2 op/s 2024-09-10T19:06:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:34 smithi107 ceph-mon[29404]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 627 B/s wr, 2 op/s 2024-09-10T19:06:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:34 smithi107 ceph-mon[40355]: pgmap v893: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 627 B/s wr, 2 op/s 2024-09-10T19:06:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:36 smithi162 ceph-mon[33228]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:36 smithi107 ceph-mon[29404]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:36.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:36 smithi107 ceph-mon[40355]: pgmap v894: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:06:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:38 smithi162 ceph-mon[33228]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:06:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:38 smithi162 ceph-mon[33228]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T19:06:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[29404]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:06:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[29404]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T19:06:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[40355]: pgmap v895: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:06:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[40355]: osdmap e576: 8 total, 8 up, 8 in 2024-09-10T19:06:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:39.363 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:39] "GET /metrics HTTP/1.1" 200 228663 "" "Prometheus/2.43.0" 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:39 smithi162 ceph-mon[33228]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T19:06:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T19:06:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[29404]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: osdmap e577: 8 total, 8 up, 8 in 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]: dispatch 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-116"}]': finished 2024-09-10T19:06:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:39 smithi107 ceph-mon[40355]: osdmap e578: 8 total, 8 up, 8 in 2024-09-10T19:06:40.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:40 smithi162 ceph-mon[33228]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-10T19:06:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:40 smithi107 ceph-mon[29404]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-10T19:06:40.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:40 smithi107 ceph-mon[40355]: pgmap v898: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-10T19:06:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:41 smithi162 ceph-mon[33228]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T19:06:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:41 smithi162 ceph-mon[33228]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T19:06:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[29404]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T19:06:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[29404]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T19:06:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:41.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[40355]: osdmap e579: 8 total, 8 up, 8 in 2024-09-10T19:06:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[40355]: osdmap e580: 8 total, 8 up, 8 in 2024-09-10T19:06:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:41.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:42.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:42 smithi162 ceph-mon[33228]: pgmap v901: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:42 smithi162 ceph-mon[33228]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T19:06:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[29404]: pgmap v901: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[29404]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T19:06:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[40355]: pgmap v901: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-118","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:42 smithi107 ceph-mon[40355]: osdmap e581: 8 total, 8 up, 8 in 2024-09-10T19:06:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: pgmap v904: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T19:06:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:44.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T19:06:44.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:44.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: pgmap v904: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: pgmap v904: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: osdmap e582: 8 total, 8 up, 8 in 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: osdmap e583: 8 total, 8 up, 8 in 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:44.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:46.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:46.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: pgmap v908: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: pgmap v908: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:06:46.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:46.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: osdmap e584: 8 total, 8 up, 8 in 2024-09-10T19:06:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]: dispatch 2024-09-10T19:06:46.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: pgmap v908: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:06:46.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:46.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:06:47.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:47.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]': finished 2024-09-10T19:06:47.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[29404]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T19:06:47.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:47.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]': finished 2024-09-10T19:06:47.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:47 smithi107 ceph-mon[40355]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T19:06:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:47 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-118", "mode": "writeback"}]': finished 2024-09-10T19:06:47.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:47 smithi162 ceph-mon[33228]: osdmap e585: 8 total, 8 up, 8 in 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:48.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:48.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:48.467 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:48.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T19:06:48.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-10T19:06:48.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:48.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:48.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:48.468 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: osdmap e586: 8 total, 8 up, 8 in 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: pgmap v911: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 524 B/s wr, 1 op/s 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:06:48.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:06:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:49] "GET /metrics HTTP/1.1" 200 228632 "" "Prometheus/2.43.0" 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[29404]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[40355]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:49.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:49 smithi162 ceph-mon[33228]: osdmap e587: 8 total, 8 up, 8 in 2024-09-10T19:06:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:49.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]: dispatch 2024-09-10T19:06:50.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[29404]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[29404]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[40355]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T19:06:50.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:50 smithi107 ceph-mon[40355]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:06:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:50 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-118"}]': finished 2024-09-10T19:06:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:50 smithi162 ceph-mon[33228]: osdmap e588: 8 total, 8 up, 8 in 2024-09-10T19:06:50.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:50 smithi162 ceph-mon[33228]: pgmap v914: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:06:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:06:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:06:51.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:51 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:51.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:51 smithi107 ceph-mon[29404]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T19:06:51.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:51 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:51.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:51 smithi107 ceph-mon[40355]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T19:06:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:51 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:51.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:51 smithi162 ceph-mon[33228]: osdmap e589: 8 total, 8 up, 8 in 2024-09-10T19:06:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:52 smithi162 ceph-mon[33228]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T19:06:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:52 smithi162 ceph-mon[33228]: pgmap v917: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[29404]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T19:06:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[29404]: pgmap v917: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[40355]: osdmap e590: 8 total, 8 up, 8 in 2024-09-10T19:06:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:52.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:52 smithi107 ceph-mon[40355]: pgmap v917: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:06:53.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:06:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:06:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:53 smithi162 ceph-mon[33228]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T19:06:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:53.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:53.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[29404]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T19:06:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:53.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-120","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:06:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[40355]: osdmap e591: 8 total, 8 up, 8 in 2024-09-10T19:06:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:53.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:54 smithi162 ceph-mon[33228]: pgmap v920: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[29404]: pgmap v920: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: osdmap e592: 8 total, 8 up, 8 in 2024-09-10T19:06:54.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:54.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:54 smithi107 ceph-mon[40355]: pgmap v920: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:55 smithi162 ceph-mon[33228]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T19:06:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:55.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[29404]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[40355]: osdmap e593: 8 total, 8 up, 8 in 2024-09-10T19:06:55.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:55.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]: dispatch 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]': finished 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: pgmap v923: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:56.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:56.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]': finished 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: pgmap v923: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-120", "mode": "writeback"}]': finished 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: osdmap e594: 8 total, 8 up, 8 in 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: pgmap v923: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:06:56.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:56.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:06:57.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:57.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:57 smithi162 ceph-mon[33228]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T19:06:57.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:57.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[29404]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T19:06:57.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:57.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:06:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[40355]: osdmap e595: 8 total, 8 up, 8 in 2024-09-10T19:06:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:57.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]: dispatch 2024-09-10T19:06:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:58 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:58 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:58 smithi162 ceph-mon[33228]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T19:06:58.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:58 smithi162 ceph-mon[33228]: pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:06:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[29404]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T19:06:58.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[29404]: pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:06:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:06:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-120"}]': finished 2024-09-10T19:06:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[40355]: osdmap e596: 8 total, 8 up, 8 in 2024-09-10T19:06:58.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:58 smithi107 ceph-mon[40355]: pgmap v926: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 3 op/s 2024-09-10T19:06:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:06:59] "GET /metrics HTTP/1.1" 200 228632 "" "Prometheus/2.43.0" 2024-09-10T19:06:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[29404]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T19:06:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[29404]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T19:06:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:59.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[40355]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T19:06:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[40355]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T19:06:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:59.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:06:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:59 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:06:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:59 smithi162 ceph-mon[33228]: osdmap e597: 8 total, 8 up, 8 in 2024-09-10T19:06:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:59 smithi162 ceph-mon[33228]: osdmap e598: 8 total, 8 up, 8 in 2024-09-10T19:06:59.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:06:59.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:06:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-mon[33228]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:07:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-mon[33228]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T19:07:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:00.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[29404]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:07:00.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[29404]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[40355]: pgmap v929: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-122","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[40355]: osdmap e599: 8 total, 8 up, 8 in 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:00.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:01 smithi162 ceph-mon[33228]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T19:07:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:01.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:01.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[29404]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[40355]: osdmap e600: 8 total, 8 up, 8 in 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:01.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-mon[33228]: pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-mon[33228]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T19:07:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:02.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[29404]: pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[29404]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T19:07:02.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[40355]: pgmap v932: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[40355]: osdmap e601: 8 total, 8 up, 8 in 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:02.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]: dispatch 2024-09-10T19:07:03.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:03 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]': finished 2024-09-10T19:07:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:03 smithi162 ceph-mon[33228]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T19:07:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:03 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:03.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]': finished 2024-09-10T19:07:03.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[29404]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-122", "mode": "writeback"}]': finished 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[40355]: osdmap e602: 8 total, 8 up, 8 in 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:03.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:04.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:04.737 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: pgmap v935: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:04.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:04.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:04.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T19:07:04.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:04.738 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: pgmap v935: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: pgmap v935: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:04.739 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:04.740 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:04.740 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: osdmap e603: 8 total, 8 up, 8 in 2024-09-10T19:07:04.740 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:04.740 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]: dispatch 2024-09-10T19:07:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:05 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:05 smithi162 ceph-mon[33228]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T19:07:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:05.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[29404]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T19:07:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-122"}]': finished 2024-09-10T19:07:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:05 smithi107 ceph-mon[40355]: osdmap e604: 8 total, 8 up, 8 in 2024-09-10T19:07:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:06 smithi107 ceph-mon[29404]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:06 smithi107 ceph-mon[29404]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T19:07:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:06 smithi107 ceph-mon[40355]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:06 smithi107 ceph-mon[40355]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T19:07:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:06 smithi162 ceph-mon[33228]: pgmap v938: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:06 smithi162 ceph-mon[33228]: osdmap e605: 8 total, 8 up, 8 in 2024-09-10T19:07:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:08 smithi162 ceph-mon[33228]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T19:07:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:08 smithi162 ceph-mon[33228]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[29404]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[29404]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[40355]: osdmap e606: 8 total, 8 up, 8 in 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:08 smithi107 ceph-mon[40355]: pgmap v941: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[29404]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T19:07:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:09.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:09] "GET /metrics HTTP/1.1" 200 228662 "" "Prometheus/2.43.0" 2024-09-10T19:07:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[40355]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T19:07:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:09 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-124","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:09 smithi162 ceph-mon[33228]: osdmap e607: 8 total, 8 up, 8 in 2024-09-10T19:07:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:09 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:10 smithi162 ceph-mon[33228]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T19:07:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:10 smithi162 ceph-mon[33228]: pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[29404]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[29404]: pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[40355]: osdmap e608: 8 total, 8 up, 8 in 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:10 smithi107 ceph-mon[40355]: pgmap v944: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:11 smithi162 ceph-mon[33228]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T19:07:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[29404]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T19:07:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[40355]: osdmap e609: 8 total, 8 up, 8 in 2024-09-10T19:07:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:12.575 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-mon[33228]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T19:07:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-mon[33228]: pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[29404]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T19:07:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[29404]: pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[40355]: osdmap e610: 8 total, 8 up, 8 in 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]: dispatch 2024-09-10T19:07:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:12 smithi107 ceph-mon[40355]: pgmap v947: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:12.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:13.595 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:13 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:13.595 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]': finished 2024-09-10T19:07:13.595 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:13 smithi162 ceph-mon[33228]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T19:07:13.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]': finished 2024-09-10T19:07:13.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[29404]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T19:07:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-124", "mode": "writeback"}]': finished 2024-09-10T19:07:13.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:13 smithi107 ceph-mon[40355]: osdmap e611: 8 total, 8 up, 8 in 2024-09-10T19:07:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T19:07:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:14.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:14 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: osdmap e612: 8 total, 8 up, 8 in 2024-09-10T19:07:14.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:14.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: pgmap v950: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:14.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:14 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:15 smithi162 ceph-mon[33228]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T19:07:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[29404]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T19:07:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[40355]: osdmap e613: 8 total, 8 up, 8 in 2024-09-10T19:07:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]: dispatch 2024-09-10T19:07:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:16 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:16 smithi162 ceph-mon[33228]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T19:07:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:16 smithi162 ceph-mon[33228]: pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[29404]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[29404]: pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-124"}]': finished 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[40355]: osdmap e614: 8 total, 8 up, 8 in 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[40355]: pgmap v953: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.7 KiB/s wr, 3 op/s 2024-09-10T19:07:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:17 smithi162 ceph-mon[33228]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T19:07:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:17 smithi107 ceph-mon[29404]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T19:07:17.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:17 smithi107 ceph-mon[40355]: osdmap e615: 8 total, 8 up, 8 in 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T19:07:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:18.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T19:07:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: osdmap e616: 8 total, 8 up, 8 in 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: pgmap v956: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-126","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: osdmap e617: 8 total, 8 up, 8 in 2024-09-10T19:07:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:19.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[29404]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:19] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:19.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:19.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[40355]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T19:07:19.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:19.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:19 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:19 smithi162 ceph-mon[33228]: osdmap e618: 8 total, 8 up, 8 in 2024-09-10T19:07:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-mon[33228]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-mon[33228]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T19:07:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[29404]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[29404]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[40355]: pgmap v959: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[40355]: osdmap e619: 8 total, 8 up, 8 in 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]: dispatch 2024-09-10T19:07:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:21 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]': finished 2024-09-10T19:07:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:21 smithi162 ceph-mon[33228]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T19:07:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:21.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]': finished 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[29404]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-126", "mode": "writeback"}]': finished 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[40355]: osdmap e620: 8 total, 8 up, 8 in 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-mon[33228]: pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-mon[33228]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T19:07:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[29404]: pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[29404]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[40355]: pgmap v962: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[40355]: osdmap e621: 8 total, 8 up, 8 in 2024-09-10T19:07:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:22.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]: dispatch 2024-09-10T19:07:23.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:23 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:23 smithi162 ceph-mon[33228]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T19:07:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:23.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[29404]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T19:07:23.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:23.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-126"}]': finished 2024-09-10T19:07:23.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:23 smithi107 ceph-mon[40355]: osdmap e622: 8 total, 8 up, 8 in 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringFlush 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringFlush (9083 ms) 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestSnapHasChunk 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestSnapHasChunk (6049 ms) 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollback 2024-09-10T19:07:24.211 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollback (5036 ms) 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestRollbackRefcount 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestRollbackRefcount (25605 ms) 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.ManifestEvictRollback 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.ManifestEvictRollback (14054 ms) 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.PropagateBaseTierError 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.PropagateBaseTierError (12097 ms) 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.HelloWriteReturn 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000000 79 6f 75 20 6d 69 67 68 74 20 73 65 65 20 74 68 |you might see th| 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000010 69 73 |is| 2024-09-10T19:07:24.212 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 00000012 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.HelloWriteReturn (12093 ms) 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: require_osd_release = quincy 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsPP.TierFlushDuringUnsetDedupTier (6027 ms) 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 48 tests from LibRadosTwoPoolsPP (555638 ms total) 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.Dirty 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.Dirty (1026 ms) 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.FlushWriteRaces 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.FlushWriteRaces (11017 ms) 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.CallForcesPromote 2024-09-10T19:07:24.213 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.CallForcesPromote (18065 ms) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTierECPP.HitSetNone 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTierECPP.HitSetNone (3 ms) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 4 tests from LibRadosTierECPP (30111 ms total) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Overlay 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Overlay (7055 ms) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Promote 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Promote (8059 ms) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnap 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: waiting for scrub... 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: done waiting 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnap (24523 ms) 2024-09-10T19:07:24.214 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteSnapTrimRace (9843 ms) 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Whiteout 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Whiteout (8074 ms) 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Evict 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Evict (7922 ms) 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.EvictSnap 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.EvictSnap (10062 ms) 2024-09-10T19:07:24.215 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlush 2024-09-10T19:07:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:24 smithi162 ceph-mon[33228]: pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:07:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:24 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:24 smithi162 ceph-mon[33228]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T19:07:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[29404]: pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:07:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[29404]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T19:07:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[40355]: pgmap v965: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 2 op/s 2024-09-10T19:07:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:24 smithi107 ceph-mon[40355]: osdmap e623: 8 total, 8 up, 8 in 2024-09-10T19:07:26.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[29404]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T19:07:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[29404]: pgmap v968: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:26.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[40355]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T19:07:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:26 smithi107 ceph-mon[40355]: pgmap v968: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:26 smithi162 ceph-mon[33228]: osdmap e624: 8 total, 8 up, 8 in 2024-09-10T19:07:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:26 smithi162 ceph-mon[33228]: pgmap v968: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:27 smithi162 ceph-mon[33228]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T19:07:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:27 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:27.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:27 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:27.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[29404]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-128","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[40355]: osdmap e625: 8 total, 8 up, 8 in 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:27.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:27 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:28 smithi162 ceph-mon[33228]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T19:07:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:28 smithi162 ceph-mon[33228]: pgmap v971: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:07:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[29404]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[29404]: pgmap v971: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[40355]: osdmap e626: 8 total, 8 up, 8 in 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:28 smithi107 ceph-mon[40355]: pgmap v971: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 op/s 2024-09-10T19:07:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:29] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]': finished 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[29404]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T19:07:29.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]': finished 2024-09-10T19:07:29.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:29 smithi107 ceph-mon[40355]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: osdmap e627: 8 total, 8 up, 8 in 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]: dispatch 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-128", "mode": "writeback"}]': finished 2024-09-10T19:07:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:29 smithi162 ceph-mon[33228]: osdmap e628: 8 total, 8 up, 8 in 2024-09-10T19:07:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:30 smithi162 ceph-mon[33228]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:30 smithi107 ceph-mon[29404]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:30.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:30 smithi107 ceph-mon[40355]: pgmap v974: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:32 smithi162 ceph-mon[33228]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-10T19:07:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[29404]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-10T19:07:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:32.467 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:32 smithi107 ceph-mon[40355]: pgmap v975: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1007 B/s rd, 0 op/s 2024-09-10T19:07:33.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:34 smithi162 ceph-mon[33228]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-10T19:07:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:34 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:34 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:34.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[40355]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[29404]: pgmap v976: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 336 B/s wr, 1 op/s 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:34 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:35 smithi162 ceph-mon[33228]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T19:07:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:35.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[29404]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[40355]: osdmap e629: 8 total, 8 up, 8 in 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]: dispatch 2024-09-10T19:07:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:36 smithi162 ceph-mon[33228]: pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-10T19:07:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:36 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:36 smithi162 ceph-mon[33228]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T19:07:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[29404]: pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[29404]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[40355]: pgmap v978: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 868 B/s rd, 289 B/s wr, 1 op/s 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-128"}]': finished 2024-09-10T19:07:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:36 smithi107 ceph-mon[40355]: osdmap e630: 8 total, 8 up, 8 in 2024-09-10T19:07:37.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:37 smithi162 ceph-mon[33228]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T19:07:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:37 smithi107 ceph-mon[29404]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T19:07:37.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:37 smithi107 ceph-mon[40355]: osdmap e631: 8 total, 8 up, 8 in 2024-09-10T19:07:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:38 smithi162 ceph-mon[33228]: pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:38 smithi162 ceph-mon[33228]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T19:07:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[29404]: pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[29404]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[40355]: pgmap v981: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[40355]: osdmap e632: 8 total, 8 up, 8 in 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:39.405 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:39] "GET /metrics HTTP/1.1" 200 228664 "" "Prometheus/2.43.0" 2024-09-10T19:07:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:39 smithi162 ceph-mon[33228]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T19:07:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:39 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:39 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[29404]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-130","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[40355]: osdmap e633: 8 total, 8 up, 8 in 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:39 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-mon[33228]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-mon[33228]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T19:07:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[29404]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[29404]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[40355]: pgmap v984: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[40355]: osdmap e634: 8 total, 8 up, 8 in 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[29404]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[40355]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:41 smithi162 ceph-mon[33228]: osdmap e635: 8 total, 8 up, 8 in 2024-09-10T19:07:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]: dispatch 2024-09-10T19:07:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]': finished 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]': finished 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:42.790 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: pgmap v987: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:42.790 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:42.790 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-130", "mode": "writeback"}]': finished 2024-09-10T19:07:42.790 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: osdmap e636: 8 total, 8 up, 8 in 2024-09-10T19:07:42.790 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:42.791 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:43.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[29404]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T19:07:43.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[40355]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T19:07:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:43 smithi162 ceph-mon[33228]: osdmap e637: 8 total, 8 up, 8 in 2024-09-10T19:07:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]: dispatch 2024-09-10T19:07:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[29404]: pgmap v990: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[29404]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[40355]: pgmap v990: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:44 smithi107 ceph-mon[40355]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T19:07:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:44 smithi162 ceph-mon[33228]: pgmap v990: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-10T19:07:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:44 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:44 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-130"}]': finished 2024-09-10T19:07:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:44 smithi162 ceph-mon[33228]: osdmap e638: 8 total, 8 up, 8 in 2024-09-10T19:07:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:45 smithi162 ceph-mon[33228]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T19:07:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:45 smithi107 ceph-mon[29404]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T19:07:45.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:45 smithi107 ceph-mon[40355]: osdmap e639: 8 total, 8 up, 8 in 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: pgmap v993: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: pgmap v993: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T19:07:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: pgmap v993: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: osdmap e640: 8 total, 8 up, 8 in 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:07:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:47 smithi162 ceph-mon[33228]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T19:07:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[40355]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T19:07:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-132","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[29404]: osdmap e641: 8 total, 8 up, 8 in 2024-09-10T19:07:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:47.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:48 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:48.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:48.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: pgmap v996: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: osdmap e642: 8 total, 8 up, 8 in 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:48.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:48 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:07:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:49] "GET /metrics HTTP/1.1" 200 228668 "" "Prometheus/2.43.0" 2024-09-10T19:07:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T19:07:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T19:07:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: osdmap e643: 8 total, 8 up, 8 in 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:07:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:07:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:50 smithi162 ceph-mon[33228]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:50 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]': finished 2024-09-10T19:07:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:50 smithi162 ceph-mon[33228]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T19:07:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[29404]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]': finished 2024-09-10T19:07:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[29404]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T19:07:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[40355]: pgmap v999: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:07:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:07:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-132", "mode": "writeback"}]': finished 2024-09-10T19:07:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:50 smithi107 ceph-mon[40355]: osdmap e644: 8 total, 8 up, 8 in 2024-09-10T19:07:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:07:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:07:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:51 smithi162 ceph-mon[33228]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T19:07:51.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:51 smithi107 ceph-mon[29404]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T19:07:51.968 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:51 smithi107 ceph-mon[40355]: osdmap e645: 8 total, 8 up, 8 in 2024-09-10T19:07:52.801 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:52 smithi162 ceph-mon[33228]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:52.801 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:52 smithi162 ceph-mon[33228]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T19:07:52.801 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:52.801 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[29404]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[29404]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T19:07:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[40355]: pgmap v1002: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:07:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[40355]: osdmap e646: 8 total, 8 up, 8 in 2024-09-10T19:07:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:53.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:07:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:07:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:53 smithi162 ceph-mon[33228]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T19:07:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[29404]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T19:07:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[40355]: osdmap e647: 8 total, 8 up, 8 in 2024-09-10T19:07:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:54 smithi162 ceph-mon[33228]: pgmap v1005: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:54 smithi162 ceph-mon[33228]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T19:07:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[29404]: pgmap v1005: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[29404]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[40355]: pgmap v1005: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:54 smithi107 ceph-mon[40355]: osdmap e648: 8 total, 8 up, 8 in 2024-09-10T19:07:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:55 smithi162 ceph-mon[33228]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T19:07:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[29404]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T19:07:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:55.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[40355]: osdmap e649: 8 total, 8 up, 8 in 2024-09-10T19:07:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:07:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:56 smithi162 ceph-mon[33228]: pgmap v1008: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:56 smithi162 ceph-mon[33228]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T19:07:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[29404]: pgmap v1008: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[29404]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[40355]: pgmap v1008: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 1.5 KiB/s wr, 3 op/s 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[40355]: osdmap e650: 8 total, 8 up, 8 in 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]: dispatch 2024-09-10T19:07:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:57 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:57 smithi162 ceph-mon[33228]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T19:07:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[29404]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T19:07:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:07:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-132"}]': finished 2024-09-10T19:07:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:57 smithi107 ceph-mon[40355]: osdmap e651: 8 total, 8 up, 8 in 2024-09-10T19:07:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:58 smithi162 ceph-mon[33228]: pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:07:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:58 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:58 smithi162 ceph-mon[33228]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T19:07:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[29404]: pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:07:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:58.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[29404]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T19:07:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[40355]: pgmap v1011: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.0 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:07:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:07:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:58 smithi107 ceph-mon[40355]: osdmap e652: 8 total, 8 up, 8 in 2024-09-10T19:07:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:07:59] "GET /metrics HTTP/1.1" 200 228668 "" "Prometheus/2.43.0" 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:07:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:59.969 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: osdmap e653: 8 total, 8 up, 8 in 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-134","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: osdmap e654: 8 total, 8 up, 8 in 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:07:59.970 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:07:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-mon[33228]: pgmap v1015: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-mon[33228]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[29404]: pgmap v1015: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[29404]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[40355]: pgmap v1015: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[40355]: osdmap e655: 8 total, 8 up, 8 in 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:01 smithi162 ceph-mon[33228]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T19:08:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[29404]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[40355]: osdmap e656: 8 total, 8 up, 8 in 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]: dispatch 2024-09-10T19:08:02.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]': finished 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]': finished 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: pgmap v1018: 204 pgs: 32 unknown, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-134", "mode": "writeback"}]': finished 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: osdmap e657: 8 total, 8 up, 8 in 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:04.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: pgmap v1021: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:04.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:04.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: pgmap v1021: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: osdmap e658: 8 total, 8 up, 8 in 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]: dispatch 2024-09-10T19:08:04.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: pgmap v1021: 204 pgs: 204 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:04.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:05 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:05 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:05.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:05 smithi162 ceph-mon[33228]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T19:08:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:05.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[29404]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T19:08:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-134"}]': finished 2024-09-10T19:08:05.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:05 smithi107 ceph-mon[40355]: osdmap e659: 8 total, 8 up, 8 in 2024-09-10T19:08:06.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:06 smithi107 ceph-mon[29404]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T19:08:06.617 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:06 smithi107 ceph-mon[29404]: pgmap v1024: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:06.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:06 smithi107 ceph-mon[40355]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T19:08:06.617 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:06 smithi107 ceph-mon[40355]: pgmap v1024: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:06 smithi162 ceph-mon[33228]: osdmap e660: 8 total, 8 up, 8 in 2024-09-10T19:08:06.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:06 smithi162 ceph-mon[33228]: pgmap v1024: 172 pgs: 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:07 smithi162 ceph-mon[33228]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T19:08:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:07.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:07 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:07.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[40355]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T19:08:07.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:07.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:07.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[29404]: osdmap e661: 8 total, 8 up, 8 in 2024-09-10T19:08:07.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:07.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:07 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:08 smithi162 ceph-mon[33228]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T19:08:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:08 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:08 smithi162 ceph-mon[33228]: pgmap v1027: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:08.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[29404]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[29404]: pgmap v1027: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-136","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[40355]: osdmap e662: 8 total, 8 up, 8 in 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:08.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:08 smithi107 ceph-mon[40355]: pgmap v1027: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[29404]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T19:08:09.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:09] "GET /metrics HTTP/1.1" 200 228692 "" "Prometheus/2.43.0" 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[40355]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:09.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:09 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:09 smithi162 ceph-mon[33228]: osdmap e663: 8 total, 8 up, 8 in 2024-09-10T19:08:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:09.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:09 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-mon[33228]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T19:08:10.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-mon[33228]: pgmap v1030: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[29404]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[29404]: pgmap v1030: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[40355]: osdmap e664: 8 total, 8 up, 8 in 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]: dispatch 2024-09-10T19:08:10.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:10 smithi107 ceph-mon[40355]: pgmap v1030: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s 2024-09-10T19:08:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:11 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]': finished 2024-09-10T19:08:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:11 smithi162 ceph-mon[33228]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T19:08:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:11.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]': finished 2024-09-10T19:08:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[29404]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T19:08:11.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-136", "mode": "writeback"}]': finished 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[40355]: osdmap e665: 8 total, 8 up, 8 in 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:11.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: pgmap v1033: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:12.576 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:12.577 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-mon[33228]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T19:08:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T19:08:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: pgmap v1033: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[29404]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: osdmap e666: 8 total, 8 up, 8 in 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]: dispatch 2024-09-10T19:08:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: pgmap v1033: 204 pgs: 32 creating+peering, 172 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-136"}]': finished 2024-09-10T19:08:12.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:12 smithi107 ceph-mon[40355]: osdmap e667: 8 total, 8 up, 8 in 2024-09-10T19:08:12.924 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:14 smithi162 ceph-mon[33228]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T19:08:14.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:14 smithi162 ceph-mon[33228]: pgmap v1036: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[29404]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T19:08:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[29404]: pgmap v1036: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[40355]: osdmap e668: 8 total, 8 up, 8 in 2024-09-10T19:08:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:14 smithi107 ceph-mon[40355]: pgmap v1036: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:15 smithi162 ceph-mon[33228]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T19:08:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:15.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[29404]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T19:08:15.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:15.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[40355]: osdmap e669: 8 total, 8 up, 8 in 2024-09-10T19:08:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:15.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T19:08:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-138","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: osdmap e670: 8 total, 8 up, 8 in 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: pgmap v1039: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 977 KiB/s wr, 1 op/s 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:17 smithi162 ceph-mon[33228]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T19:08:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:17.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:17.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[29404]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[40355]: osdmap e671: 8 total, 8 up, 8 in 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:17.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T19:08:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:18.675 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:18 smithi162 ceph-mon[33228]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T19:08:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:18.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[29404]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: pgmap v1041: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: osdmap e672: 8 total, 8 up, 8 in 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:18.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:18 smithi107 ceph-mon[40355]: osdmap e673: 8 total, 8 up, 8 in 2024-09-10T19:08:19.328 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:19] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-10T19:08:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:08:19.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:19 smithi162 ceph-mon[33228]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T19:08:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:08:19.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[29404]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T19:08:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]: dispatch 2024-09-10T19:08:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-138","var": "hit_set_type","val": "explicit_object"}]': finished 2024-09-10T19:08:19.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:19 smithi107 ceph-mon[40355]: osdmap e674: 8 total, 8 up, 8 in 2024-09-10T19:08:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:20 smithi162 ceph-mon[33228]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:20 smithi107 ceph-mon[29404]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:20 smithi107 ceph-mon[40355]: pgmap v1044: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:21.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:21.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:21.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]: dispatch 2024-09-10T19:08:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:22 smithi162 ceph-mon[33228]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]': finished 2024-09-10T19:08:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:22 smithi162 ceph-mon[33228]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T19:08:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[29404]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]': finished 2024-09-10T19:08:22.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[29404]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T19:08:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[40355]: pgmap v1046: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:08:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-138"}]': finished 2024-09-10T19:08:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:22 smithi107 ceph-mon[40355]: osdmap e675: 8 total, 8 up, 8 in 2024-09-10T19:08:23.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:23.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:23 smithi162 ceph-mon[33228]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T19:08:23.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:23 smithi107 ceph-mon[29404]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T19:08:23.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:23 smithi107 ceph-mon[40355]: osdmap e676: 8 total, 8 up, 8 in 2024-09-10T19:08:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:24 smithi162 ceph-mon[33228]: pgmap v1049: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:24 smithi162 ceph-mon[33228]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T19:08:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:24.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[29404]: pgmap v1049: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[29404]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[40355]: pgmap v1049: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[40355]: osdmap e677: 8 total, 8 up, 8 in 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:24.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:25.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:25.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-140","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:25.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: osdmap e678: 8 total, 8 up, 8 in 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: osdmap e679: 8 total, 8 up, 8 in 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:25.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]: dispatch 2024-09-10T19:08:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:26 smithi162 ceph-mon[33228]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T19:08:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:26 smithi162 ceph-mon[33228]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T19:08:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:26 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[29404]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:26.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[29404]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[40355]: pgmap v1052: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_count","val": "3"}]': finished 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[40355]: osdmap e680: 8 total, 8 up, 8 in 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:26.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]: dispatch 2024-09-10T19:08:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:28 smithi162 ceph-mon[33228]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T19:08:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:28 smithi162 ceph-mon[33228]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T19:08:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:28 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:28 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[29404]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T19:08:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[29404]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T19:08:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:28.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[40355]: pgmap v1055: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_period","val": "3"}]': finished 2024-09-10T19:08:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[40355]: osdmap e681: 8 total, 8 up, 8 in 2024-09-10T19:08:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:28.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:28 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:29.374 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:29] "GET /metrics HTTP/1.1" 200 228683 "" "Prometheus/2.43.0" 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T19:08:29.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:29 smithi162 ceph-mon[33228]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T19:08:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T19:08:29.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[29404]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: osdmap e682: 8 total, 8 up, 8 in 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]: dispatch 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-140","var": "hit_set_fpp","val": ".01"}]': finished 2024-09-10T19:08:29.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:29 smithi107 ceph-mon[40355]: osdmap e683: 8 total, 8 up, 8 in 2024-09-10T19:08:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:30 smithi162 ceph-mon[33228]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:30 smithi107 ceph-mon[29404]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:30.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:30 smithi107 ceph-mon[40355]: pgmap v1058: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:32.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:32 smithi162 ceph-mon[33228]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:32 smithi107 ceph-mon[29404]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:32 smithi107 ceph-mon[40355]: pgmap v1060: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:08:33.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:34 smithi162 ceph-mon[33228]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T19:08:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:34 smithi107 ceph-mon[29404]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T19:08:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:34 smithi107 ceph-mon[40355]: pgmap v1061: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.0 KiB/s wr, 1 op/s 2024-09-10T19:08:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:36 smithi162 ceph-mon[33228]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 643 B/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-10T19:08:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:36 smithi107 ceph-mon[29404]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 643 B/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-10T19:08:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:36 smithi107 ceph-mon[40355]: pgmap v1062: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 643 B/s rd, 3.0 KiB/s wr, 1 op/s 2024-09-10T19:08:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:38 smithi162 ceph-mon[33228]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-10T19:08:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:38 smithi107 ceph-mon[29404]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-10T19:08:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:38 smithi107 ceph-mon[40355]: pgmap v1063: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 7.4 KiB/s wr, 2 op/s 2024-09-10T19:08:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:39] "GET /metrics HTTP/1.1" 200 228687 "" "Prometheus/2.43.0" 2024-09-10T19:08:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:40 smithi162 ceph-mon[33228]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-10T19:08:40.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:40 smithi107 ceph-mon[40355]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-10T19:08:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:40 smithi107 ceph-mon[29404]: pgmap v1064: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 6.6 KiB/s wr, 1 op/s 2024-09-10T19:08:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-mon[33228]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 771 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-10T19:08:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:42.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[40355]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 771 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-10T19:08:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[29404]: pgmap v1065: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 771 B/s rd, 5.5 KiB/s wr, 1 op/s 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]: dispatch 2024-09-10T19:08:43.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]': finished 2024-09-10T19:08:43.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:43 smithi162 ceph-mon[33228]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T19:08:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]': finished 2024-09-10T19:08:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:43 smithi107 ceph-mon[40355]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T19:08:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-140"}]': finished 2024-09-10T19:08:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:43 smithi107 ceph-mon[29404]: osdmap e684: 8 total, 8 up, 8 in 2024-09-10T19:08:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:44 smithi162 ceph-mon[33228]: pgmap v1067: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-10T19:08:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:44 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:44.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:44 smithi162 ceph-mon[33228]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T19:08:44.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:44.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[40355]: pgmap v1067: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[40355]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[29404]: pgmap v1067: 204 pgs: 204 active+clean; 4.5 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 921 B/s rd, 8.4 KiB/s wr, 2 op/s 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:44 smithi107 ceph-mon[29404]: osdmap e685: 8 total, 8 up, 8 in 2024-09-10T19:08:45.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[40355]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T19:08:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:45.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[29404]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T19:08:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:45.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:45 smithi162 ceph-mon[33228]: osdmap e686: 8 total, 8 up, 8 in 2024-09-10T19:08:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: pgmap v1070: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:46.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: pgmap v1070: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:46.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:46.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: pgmap v1070: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-142","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: osdmap e687: 8 total, 8 up, 8 in 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:08:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:08:47.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:47.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[29404]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[40355]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:47.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:47 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:08:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:47 smithi162 ceph-mon[33228]: osdmap e688: 8 total, 8 up, 8 in 2024-09-10T19:08:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:47 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:47 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[29404]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:08:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[29404]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[40355]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[40355]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:48 smithi162 ceph-mon[33228]: pgmap v1073: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:08:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:48 smithi162 ceph-mon[33228]: osdmap e689: 8 total, 8 up, 8 in 2024-09-10T19:08:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]: dispatch 2024-09-10T19:08:49.442 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:49] "GET /metrics HTTP/1.1" 200 228634 "" "Prometheus/2.43.0" 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]': finished 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:08:49.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]': finished 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:08:49.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:49 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-142", "mode": "writeback"}]': finished 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: osdmap e690: 8 total, 8 up, 8 in 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:08:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:49 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[29404]: pgmap v1076: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:08:50.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[29404]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[40355]: pgmap v1076: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[40355]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:50.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:50 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-mon[33228]: pgmap v1076: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 5.0 KiB/s wr, 6 op/s 2024-09-10T19:08:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:08:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-mon[33228]: osdmap e691: 8 total, 8 up, 8 in 2024-09-10T19:08:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:08:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:08:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:08:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[29404]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T19:08:51.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:51.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[40355]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T19:08:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:51.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:08:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:51 smithi162 ceph-mon[33228]: osdmap e692: 8 total, 8 up, 8 in 2024-09-10T19:08:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:52.865 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: pgmap v1079: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: osdmap e693: 8 total, 8 up, 8 in 2024-09-10T19:08:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:08:53.173 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:08:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:08:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:08:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T19:08:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:08:53.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:53 smithi162 ceph-mon[33228]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T19:08:53.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:08:53.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T19:08:53.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[40355]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: osdmap e694: 8 total, 8 up, 8 in 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]: dispatch 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_grade_decay_rate","val": "20"}]': finished 2024-09-10T19:08:53.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:53 smithi107 ceph-mon[29404]: osdmap e695: 8 total, 8 up, 8 in 2024-09-10T19:08:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:08:54.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:54 smithi162 ceph-mon[33228]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T19:08:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[29404]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: pgmap v1082: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]: dispatch 2024-09-10T19:08:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-142","var": "hit_set_search_last_n","val": "1"}]': finished 2024-09-10T19:08:54.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:54 smithi107 ceph-mon[40355]: osdmap e696: 8 total, 8 up, 8 in 2024-09-10T19:08:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:56 smithi162 ceph-mon[33228]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:56 smithi162 ceph-mon[33228]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T19:08:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[29404]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[29404]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[40355]: pgmap v1085: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[40355]: osdmap e697: 8 total, 8 up, 8 in 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]': finished 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: osdmap e698: 8 total, 8 up, 8 in 2024-09-10T19:08:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:08:57.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:57.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-142"}]: dispatch 2024-09-10T19:08:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:58 smithi162 ceph-mon[33228]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:08:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:58 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:58 smithi162 ceph-mon[33228]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T19:08:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[29404]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:08:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[29404]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T19:08:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[40355]: pgmap v1088: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:08:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:08:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:58 smithi107 ceph-mon[40355]: osdmap e699: 8 total, 8 up, 8 in 2024-09-10T19:08:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:08:59] "GET /metrics HTTP/1.1" 200 228634 "" "Prometheus/2.43.0" 2024-09-10T19:08:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:59 smithi162 ceph-mon[33228]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T19:08:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:08:59 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[29404]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T19:08:59.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[40355]: osdmap e700: 8 total, 8 up, 8 in 2024-09-10T19:08:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:08:59.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:08:59 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:00.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-mon[33228]: pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:09:00.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:00.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-mon[33228]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T19:09:00.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:00.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[40355]: pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:09:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[40355]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T19:09:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[29404]: pgmap v1091: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 1 op/s 2024-09-10T19:09:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-144","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[29404]: osdmap e701: 8 total, 8 up, 8 in 2024-09-10T19:09:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:00.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:00 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:01 smithi162 ceph-mon[33228]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T19:09:01.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:01 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[40355]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T19:09:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:01.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:01.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[29404]: osdmap e702: 8 total, 8 up, 8 in 2024-09-10T19:09:01.967 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:01.968 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:02.875 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-mon[33228]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:02.875 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:02.875 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-mon[33228]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T19:09:02.875 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:02.875 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[40355]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:02.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[40355]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[29404]: pgmap v1094: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[29404]: osdmap e703: 8 total, 8 up, 8 in 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:02.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:02 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]: dispatch 2024-09-10T19:09:03.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:03 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:03 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]': finished 2024-09-10T19:09:03.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:03 smithi162 ceph-mon[33228]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T19:09:03.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:03.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]': finished 2024-09-10T19:09:03.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[40355]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T19:09:03.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:03.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-144", "mode": "readproxy"}]': finished 2024-09-10T19:09:03.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:03 smithi107 ceph-mon[29404]: osdmap e704: 8 total, 8 up, 8 in 2024-09-10T19:09:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:04 smithi162 ceph-mon[33228]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:09:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:04 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:04.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:04.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[40355]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:09:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[29404]: pgmap v1097: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 255 B/s wr, 1 op/s 2024-09-10T19:09:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:04 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:06 smithi162 ceph-mon[33228]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 176 B/s wr, 1 op/s 2024-09-10T19:09:06.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:06 smithi107 ceph-mon[40355]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 176 B/s wr, 1 op/s 2024-09-10T19:09:06.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:06 smithi107 ceph-mon[29404]: pgmap v1098: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 882 B/s rd, 176 B/s wr, 1 op/s 2024-09-10T19:09:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:08 smithi162 ceph-mon[33228]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 150 B/s wr, 1 op/s 2024-09-10T19:09:08.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:08 smithi107 ceph-mon[40355]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 150 B/s wr, 1 op/s 2024-09-10T19:09:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:08 smithi107 ceph-mon[29404]: pgmap v1099: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.6 KiB/s rd, 150 B/s wr, 1 op/s 2024-09-10T19:09:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:09] "GET /metrics HTTP/1.1" 200 228666 "" "Prometheus/2.43.0" 2024-09-10T19:09:10.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:10 smithi162 ceph-mon[33228]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T19:09:10.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:10 smithi107 ceph-mon[40355]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T19:09:10.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:10 smithi107 ceph-mon[29404]: pgmap v1100: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.4 KiB/s rd, 127 B/s wr, 1 op/s 2024-09-10T19:09:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:12.886 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:12 smithi162 ceph-mon[33228]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-10T19:09:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:12 smithi107 ceph-mon[29404]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-10T19:09:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:12 smithi107 ceph-mon[40355]: pgmap v1101: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 104 B/s wr, 1 op/s 2024-09-10T19:09:13.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:14.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:14.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: pgmap v1102: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: osdmap e705: 8 total, 8 up, 8 in 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:15 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]': finished 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: osdmap e706: 8 total, 8 up, 8 in 2024-09-10T19:09:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:15.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:15 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-144"}]: dispatch 2024-09-10T19:09:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:16 smithi162 ceph-mon[33228]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:09:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:16 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:16 smithi162 ceph-mon[33228]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T19:09:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[29404]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:09:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[29404]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[40355]: pgmap v1105: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 639 B/s rd, 0 op/s 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[40355]: osdmap e707: 8 total, 8 up, 8 in 2024-09-10T19:09:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:17 smithi162 ceph-mon[33228]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T19:09:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:17 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[29404]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T19:09:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[40355]: osdmap e708: 8 total, 8 up, 8 in 2024-09-10T19:09:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:17 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:18 smithi162 ceph-mon[33228]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:18 smithi162 ceph-mon[33228]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T19:09:18.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:18.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[29404]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[29404]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[40355]: pgmap v1108: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-146","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[40355]: osdmap e709: 8 total, 8 up, 8 in 2024-09-10T19:09:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:18.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:19] "GET /metrics HTTP/1.1" 200 226181 "" "Prometheus/2.43.0" 2024-09-10T19:09:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:19 smithi162 ceph-mon[33228]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T19:09:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:19.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[29404]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[40355]: osdmap e710: 8 total, 8 up, 8 in 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:20.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:20.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-mon[33228]: pgmap v1111: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:20.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:20.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-mon[33228]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T19:09:20.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:20.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[29404]: pgmap v1111: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[29404]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[40355]: pgmap v1111: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier set-overlay", "pool": "test-rados-api-smithi107-80290-111", "overlaypool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[40355]: osdmap e711: 8 total, 8 up, 8 in 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]: dispatch 2024-09-10T19:09:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:21 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]': finished 2024-09-10T19:09:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:21 smithi162 ceph-mon[33228]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T19:09:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:21 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:21.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:21 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]': finished 2024-09-10T19:09:21.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[29404]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools are missing hit_sets (CACHE_POOL_NO_HIT_SET) 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier cache-mode", "pool": "test-rados-api-smithi107-80290-146", "mode": "writeback"}]': finished 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[40355]: osdmap e712: 8 total, 8 up, 8 in 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:21.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:21 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]: dispatch 2024-09-10T19:09:22.890 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-mon[33228]: pgmap v1114: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:22.891 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:09:22.891 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-mon[33228]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T19:09:22.891 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:22.891 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[29404]: pgmap v1114: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[29404]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[40355]: pgmap v1114: 204 pgs: 32 creating+peering, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_count","val": "2"}]': finished 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[40355]: osdmap e713: 8 total, 8 up, 8 in 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:22 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]: dispatch 2024-09-10T19:09:23.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:09:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:23 smithi162 ceph-mon[33228]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T19:09:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:23 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:23.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:23 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:23.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[40355]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_period","val": "600"}]': finished 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[29404]: osdmap e714: 8 total, 8 up, 8 in 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:23.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:23 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]: dispatch 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:24 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: pgmap v1117: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NO_HIT_SET (was: 1 cache pools are missing hit_sets) 2024-09-10T19:09:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "hit_set_type","val": "bloom"}]': finished 2024-09-10T19:09:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: osdmap e715: 8 total, 8 up, 8 in 2024-09-10T19:09:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]: dispatch 2024-09-10T19:09:24.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:24 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:09:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:25 smithi162 ceph-mon[33228]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T19:09:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:25 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:25.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:25 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:25.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[29404]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "min_read_recency_for_promote","val": "1"}]': finished 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[40355]: osdmap e716: 8 total, 8 up, 8 in 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:25.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:25 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]: dispatch 2024-09-10T19:09:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:26 smithi162 ceph-mon[33228]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:26 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:09:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:26 smithi162 ceph-mon[33228]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T19:09:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[29404]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:09:26.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[29404]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T19:09:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[40355]: pgmap v1120: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 2.2 KiB/s rd, 1023 B/s wr, 4 op/s 2024-09-10T19:09:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-146","var": "target_max_objects","val": "1"}]': finished 2024-09-10T19:09:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:26 smithi107 ceph-mon[40355]: osdmap e717: 8 total, 8 up, 8 in 2024-09-10T19:09:28.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:27 smithi162 ceph-mon[33228]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:09:28.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:27 smithi107 ceph-mon[29404]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:09:28.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:27 smithi107 ceph-mon[40355]: Health check failed: 1 cache pools at or near target size (CACHE_POOL_NEAR_FULL) 2024-09-10T19:09:29.066 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:28 smithi107 ceph-mon[29404]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:29.067 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:28 smithi107 ceph-mon[40355]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:29.173 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:28 smithi162 ceph-mon[33228]: pgmap v1122: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:29] "GET /metrics HTTP/1.1" 200 226181 "" "Prometheus/2.43.0" 2024-09-10T19:09:30.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:29 smithi162 ceph-mon[33228]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:30.215 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:29 smithi107 ceph-mon[29404]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:30.216 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:29 smithi107 ceph-mon[40355]: pgmap v1123: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:32 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:32.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:32 smithi162 ceph-mon[33228]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[29404]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:32.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:32 smithi107 ceph-mon[40355]: pgmap v1124: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 666 B/s rd, 0 B/s wr, 0 op/s 2024-09-10T19:09:33.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:34.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:34 smithi162 ceph-mon[33228]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:34 smithi107 ceph-mon[29404]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:34 smithi107 ceph-mon[40355]: pgmap v1125: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:36 smithi162 ceph-mon[33228]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:36.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:36.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[29404]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[40355]: pgmap v1126: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 B/s wr, 1 op/s 2024-09-10T19:09:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:37.676 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:37.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:37 smithi162 ceph-mon[33228]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T19:09:37.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:37 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:37.677 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[29404]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T19:09:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[40355]: osdmap e718: 8 total, 8 up, 8 in 2024-09-10T19:09:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:37.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:38 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:09:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T19:09:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: pgmap v1128: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]': finished 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: osdmap e719: 8 total, 8 up, 8 in 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:38.718 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:38 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-146"}]: dispatch 2024-09-10T19:09:39.419 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:39] "GET /metrics HTTP/1.1" 200 228710 "" "Prometheus/2.43.0" 2024-09-10T19:09:39.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:39 smithi162 ceph-mon[33228]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T19:09:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:39 smithi107 ceph-mon[29404]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T19:09:39.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:39 smithi107 ceph-mon[40355]: osdmap e720: 8 total, 8 up, 8 in 2024-09-10T19:09:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-mon[33228]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:09:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-mon[33228]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:09:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-mon[33228]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T19:09:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:40.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[29404]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:09:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[29404]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[29404]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[40355]: pgmap v1131: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 170 B/s rd, 0 op/s 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[40355]: Health check cleared: CACHE_POOL_NEAR_FULL (was: 1 cache pools at or near target size) 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[40355]: osdmap e721: 8 total, 8 up, 8 in 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:40 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[29404]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T19:09:41.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:41.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[40355]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T19:09:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:41.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:41 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-148","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:41 smithi162 ceph-mon[33228]: osdmap e722: 8 total, 8 up, 8 in 2024-09-10T19:09:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:41 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:41.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:41 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]: dispatch 2024-09-10T19:09:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[40355]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:42.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[40355]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[29404]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[29404]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:42.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:42 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:42.825 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-mon[33228]: pgmap v1134: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:42.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier add", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148", "force_nonempty": "--force-nonempty" }]': finished 2024-09-10T19:09:42.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-mon[33228]: osdmap e723: 8 total, 8 up, 8 in 2024-09-10T19:09:42.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:42.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]': finished 2024-09-10T19:09:43.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]': finished 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:43 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]': finished 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: osdmap e724: 8 total, 8 up, 8 in 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:43.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:43 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-148"}]: dispatch 2024-09-10T19:09:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[29404]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:09:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[29404]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[29404]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T19:09:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[40355]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:09:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[40355]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:44 smithi107 ceph-mon[40355]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T19:09:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:44 smithi162 ceph-mon[33228]: pgmap v1137: 204 pgs: 204 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 767 B/s wr, 2 op/s 2024-09-10T19:09:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:44 smithi162 ceph-mon[33228]: Health check update: 3 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:44 smithi162 ceph-mon[33228]: osdmap e725: 8 total, 8 up, 8 in 2024-09-10T19:09:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:45 smithi162 ceph-mon[33228]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T19:09:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:45 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:45.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:45 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[29404]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T19:09:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:45.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[40355]: osdmap e726: 8 total, 8 up, 8 in 2024-09-10T19:09:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:45.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:45 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:09:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T19:09:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:46.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:09:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: pgmap v1140: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 511 B/s wr, 1 op/s 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-150","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: osdmap e727: 8 total, 8 up, 8 in 2024-09-10T19:09:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-150"}]: dispatch 2024-09-10T19:09:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:46.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:09:47.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:47 smithi162 ceph-mon[33228]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T19:09:47.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:47 smithi107 ceph-mon[29404]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T19:09:47.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:47 smithi107 ceph-mon[40355]: osdmap e728: 8 total, 8 up, 8 in 2024-09-10T19:09:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:48 smithi162 ceph-mon[33228]: pgmap v1143: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:09:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:48 smithi162 ceph-mon[33228]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T19:09:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:48 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:48 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[29404]: pgmap v1143: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[29404]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[40355]: pgmap v1143: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.5 KiB/s rd, 0 B/s wr, 2 op/s 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[40355]: osdmap e729: 8 total, 8 up, 8 in 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:48 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:49] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:49.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:49 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:49.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-152","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: osdmap e730: 8 total, 8 up, 8 in 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-152"}]: dispatch 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: Health check update: 2 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:49.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:49 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: pgmap v1146: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:09:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T19:09:50.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:50.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:50.823 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: pgmap v1146: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: pgmap v1146: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: osdmap e731: 8 total, 8 up, 8 in 2024-09-10T19:09:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:09:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:09:50.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:09:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:09:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:09:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:51 smithi162 ceph-mon[33228]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T19:09:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:51 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[29404]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T19:09:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[40355]: osdmap e732: 8 total, 8 up, 8 in 2024-09-10T19:09:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:51 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]: dispatch 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.918 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.919 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.919 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.919 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: pgmap v1149: 204 pgs: 32 unknown, 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable","pool": "test-rados-api-smithi107-80290-154","app": "rados","yes_i_really_mean_it": true}]': finished 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: osdmap e733: 8 total, 8 up, 8 in 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set","pool":"test-rados-api-smithi107-80290-111","var": "dedup_tier","val": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove-overlay", "pool": "test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:52.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:52 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd tier remove", "pool": "test-rados-api-smithi107-80290-111", "tierpool": "test-rados-api-smithi107-80290-154"}]: dispatch 2024-09-10T19:09:53.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:09:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:09:53.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:53 smithi162 ceph-mon[33228]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T19:09:53.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:53 smithi107 ceph-mon[29404]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T19:09:53.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:53 smithi107 ceph-mon[40355]: osdmap e734: 8 total, 8 up, 8 in 2024-09-10T19:09:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:54 smithi162 ceph-mon[33228]: pgmap v1152: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T19:09:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:54 smithi162 ceph-mon[33228]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T19:09:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:54 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:54 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[29404]: pgmap v1152: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[29404]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[40355]: pgmap v1152: 172 pgs: 172 active+clean; 4.4 MiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.7 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[40355]: osdmap e735: 8 total, 8 up, 8 in 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:54 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlush (7936 ms) 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FailedFlush 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FailedFlush (12170 ms) 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.Flush 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.Flush (8077 ms) 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushSnap 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushSnap (13122 ms) 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.FlushTryFlushRaces 2024-09-10T19:09:55.556 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.FlushTryFlushRaces (7650 ms) 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TryFlushReadRace 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TryFlushReadRace (8061 ms) 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetRead 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: hmm, no HitSet yet 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: ok, hit_set contains 255:602f83fe:::foo:head 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetRead (9058 ms) 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.HitSetTrim 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,0 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first is 1725995310 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,0 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,0 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,0 2024-09-10T19:09:55.557 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,1725995318,1725995319,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,1725995318,1725995319,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995310,1725995312,1725995313,1725995315,1725995316,1725995318,1725995319,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: got ls 1725995313,1725995315,1725995316,1725995318,1725995319,1725995321,1725995322,0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: first now 1725995313, trimmed 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.HitSetTrim (21071 ms) 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.PromoteOn2ndRead 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: foo0 2024-09-10T19:09:55.558 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: verifying foo0 is eventually promoted 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.PromoteOn2ndRead (14092 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ProxyRead 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ProxyRead (18089 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.CachePin 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.CachePin (22806 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetRedirectRead 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetRedirectRead (5059 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.SetChunkRead 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.SetChunkRead (3018 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.ManifestPromoteRead 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.ManifestPromoteRead (3027 ms) 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ RUN ] LibRadosTwoPoolsECPP.TrySetDedupTier 2024-09-10T19:09:55.559 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ OK ] LibRadosTwoPoolsECPP.TrySetDedupTier (3024 ms) 2024-09-10T19:09:55.560 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] 22 tests from LibRadosTwoPoolsECPP (231799 ms total) 2024-09-10T19:09:55.560 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: 2024-09-10T19:09:55.560 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [----------] Global test environment tear-down 2024-09-10T19:09:55.560 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [==========] 77 tests from 4 test suites ran. (852488 ms total) 2024-09-10T19:09:55.560 INFO:tasks.workunit.client.0.smithi107.stdout: api_tier_pp: [ PASSED ] 77 tests. 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80025 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80025 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80265 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80265 2024-09-10T19:09:55.564 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80757 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80757 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80471 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80471 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80352 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80352 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80115 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80115 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.565 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80397 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80397 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80139 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80139 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80641 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80641 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79999 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 79999 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80063 2024-09-10T19:09:55.566 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80063 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80663 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80663 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79990 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 79990 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80010 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80010 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80314 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80314 2024-09-10T19:09:55.567 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80362 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80362 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80342 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80342 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80724 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80724 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80610 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80610 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80203 2024-09-10T19:09:55.568 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80203 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80500 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80500 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=79983 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 79983 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80043 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80043 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80418 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80418 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.569 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80081 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80081 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80558 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80558 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ for t in "${!pids[@]}" 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=80579 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 80579 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ exit 0 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ cleanup 2024-09-10T19:09:55.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ pkill -P 79978 2024-09-10T19:09:55.587 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-10T19:09:55.588 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T19:09:55.588 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T19:09:55.621 INFO:tasks.workunit:Running workunits matching rados/test_pool_quota.sh on client.0... 2024-09-10T19:09:55.622 INFO:tasks.workunit:Running workunit rados/test_pool_quota.sh... 2024-09-10T19:09:55.622 DEBUG:teuthology.orchestra.run.smithi107:workunit test rados/test_pool_quota.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=fad5356a94e6044ce1062d748daa5a0f2ed53a7d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rados/test_pool_quota.sh 2024-09-10T19:09:55.683 INFO:tasks.workunit.client.0.smithi107.stderr:++ uuidgen 2024-09-10T19:09:55.688 INFO:tasks.workunit.client.0.smithi107.stderr:+ p=b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:09:55.688 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool create b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 12 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- 172.21.15.107:0/3811449137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f44101f90 msgr2=0x7f5f440fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 --2- 172.21.15.107:0/3811449137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f44101f90 0x7f5f440fb650 secure :-1 s=READY pgs=2731 cs=0 l=1 rev1=1 crypto rx=0x7f5f34004890 tx=0x7f5f34009da0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- 172.21.15.107:0/3811449137 shutdown_connections 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 --2- 172.21.15.107:0/3811449137 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f441083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 --2- 172.21.15.107:0/3811449137 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f44101f90 0x7f5f440fb650 unknown :-1 s=CLOSED pgs=2731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 --2- 172.21.15.107:0/3811449137 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f441015f0 0x7f5f441019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- 172.21.15.107:0/3811449137 >> 172.21.15.107:0/3811449137 conn(0x7f5f440725f0 msgr2=0x7f5f440f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:55.772 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- 172.21.15.107:0/3811449137 shutdown_connections 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- 172.21.15.107:0/3811449137 wait complete. 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 Processor -- start 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.768+0000 7f5f4beef640 1 -- start start 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f44112ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 0x7f5f4410cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4410d2c0 con 0x7f5f44101f90 2024-09-10T19:09:55.773 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4410d430 con 0x7f5f440fbd30 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f4410d5f0 con 0x7f5f441015f0 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4a48c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 0x7f5f4410cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:38156/0 (socket says 172.21.15.107:38156) 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 -- 172.21.15.107:0/4136161686 learned_addr learned my addr 172.21.15.107:0/4136161686 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f49c8b640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f44112ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 -- 172.21.15.107:0/4136161686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 msgr2=0x7f5f44112ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:55.774 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f44112ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 -- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 msgr2=0x7f5f4410cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 0x7f5f4410cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f40005040 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f49c8b640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f44112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4a48c640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 0x7f5f4410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4948a640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f5f3402a3d0 tx=0x7f5f34009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f340098e0 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f5f34009350 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f34005040 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f34033420 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.769+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f44119930 con 0x7f5f441015f0 2024-09-10T19:09:55.775 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.770+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f440fcd70 con 0x7f5f441015f0 2024-09-10T19:09:55.776 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.771+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f5f34030020 con 0x7f5f441015f0 2024-09-10T19:09:55.776 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.771+0000 7f5f3a7fc640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 0x7f5f30071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:55.776 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.771+0000 7f5f49c8b640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 0x7f5f30071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:55.776 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.771+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(737..737 src has 1..737) v4 ==== 8680+0+0 (secure 0 0 0) 0x7f5f340aeb20 con 0x7f5f441015f0 2024-09-10T19:09:55.777 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.772+0000 7f5f49c8b640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 0x7f5f30071470 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007fd0 tx=0x7f5f40009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:55.777 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.776+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5f3407b070 con 0x7f5f441015f0 2024-09-10T19:09:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:55 smithi162 ceph-mon[33228]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T19:09:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:55 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:55 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.931 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:55.930+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12} v 0) v1 -- 0x7f5f4410b980 con 0x7f5f441015f0 2024-09-10T19:09:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[29404]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T19:09:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile rm", "name": "testprofile-test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[40355]: osdmap e736: 8 total, 8 up, 8 in 2024-09-10T19:09:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2034798967' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:55.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:55 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]: dispatch 2024-09-10T19:09:56.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.552+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]=0 pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' created v738) v1 ==== 176+0+0 (secure 0 0 0) 0x7f5f3407aa10 con 0x7f5f441015f0 2024-09-10T19:09:56.653 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.651+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12} v 0) v1 -- 0x7f5f4404bf60 con 0x7f5f441015f0 2024-09-10T19:09:56.653 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.652+0000 7f5f3a7fc640 1 -- 172.21.15.107:0/4136161686 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]=0 pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' already exists v738) v1 ==== 183+0+0 (secure 0 0 0) 0x7f5f3407abf0 con 0x7f5f441015f0 2024-09-10T19:09:56.653 INFO:tasks.workunit.client.0.smithi107.stderr:pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' already exists 2024-09-10T19:09:56.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 msgr2=0x7f5f30071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:56.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 0x7f5f30071470 secure :-1 s=READY pgs=3593 cs=0 l=1 rev1=1 crypto rx=0x7f5f40007fd0 tx=0x7f5f40009da0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 msgr2=0x7f5f44113020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:56.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 secure :-1 s=READY pgs=2732 cs=0 l=1 rev1=1 crypto rx=0x7f5f3402a3d0 tx=0x7f5f34009da0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 shutdown_connections 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5f3006efc0 0x7f5f30071470 unknown :-1 s=CLOSED pgs=3593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5f44101f90 0x7f5f4410cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5f441015f0 0x7f5f44113020 unknown :-1 s=CLOSED pgs=2732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 --2- 172.21.15.107:0/4136161686 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5f440fbd30 0x7f5f44112ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 >> 172.21.15.107:0/4136161686 conn(0x7f5f440725f0 msgr2=0x7f5f440ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 shutdown_connections 2024-09-10T19:09:56.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.654+0000 7f5f4beef640 1 -- 172.21.15.107:0/4136161686 wait complete. 2024-09-10T19:09:56.664 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_objects 10 2024-09-10T19:09:56.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 -- 172.21.15.107:0/2480812913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a740fce70 msgr2=0x7f3a74109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:56.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 --2- 172.21.15.107:0/2480812913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a740fce70 0x7f3a74109590 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c009d00 tx=0x7f3a6c004d10 comp rx=0 tx=0).stop 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 -- 172.21.15.107:0/2480812913 shutdown_connections 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 --2- 172.21.15.107:0/2480812913 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a740fce70 0x7f3a74109590 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 --2- 172.21.15.107:0/2480812913 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a74103070 0x7f3a740fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 --2- 172.21.15.107:0/2480812913 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 0x7f3a74102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 -- 172.21.15.107:0/2480812913 >> 172.21.15.107:0/2480812913 conn(0x7f3a740f82e0 msgr2=0x7f3a740fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 -- 172.21.15.107:0/2480812913 shutdown_connections 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.739+0000 7f3a79068640 1 -- 172.21.15.107:0/2480812913 wait complete. 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 Processor -- start 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 -- start start 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 0x7f3a7406ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:56.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 0x7f3a74070460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7411aec0 con 0x7f3a74103070 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7411b030 con 0x7f3a741026d0 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a79068640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7411b1f0 con 0x7f3a740fce70 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 0x7f3a74070460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a72d9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:38162/0 (socket says 172.21.15.107:38162) 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a72d9d640 1 -- 172.21.15.107:0/260501348 learned_addr learned my addr 172.21.15.107:0/260501348 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7259c640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 0x7f3a7406ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 -- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 msgr2=0x7f3a7406f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 -- 172.21.15.107:0/260501348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 msgr2=0x7f3a7406ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 0x7f3a7406ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a68005040 con 0x7f3a74103070 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a72d9d640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.740+0000 7f3a7359e640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 0x7f3a74070460 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c02a840 tx=0x7f3a6c004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c038070 con 0x7f3a74103070 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a6c0098b0 con 0x7f3a74103070 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f3a6c009510 con 0x7f3a74103070 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6c03c840 con 0x7f3a74103070 2024-09-10T19:09:56.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a7411b7b0 con 0x7f3a74103070 2024-09-10T19:09:56.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.741+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a740fde80 con 0x7f3a74103070 2024-09-10T19:09:56.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.742+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f3a6c030070 con 0x7f3a74103070 2024-09-10T19:09:56.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.743+0000 7f3a5b7fe640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 0x7f3a5c071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:56.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.743+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(738..738 src has 1..738) v4 ==== 9055+0+0 (secure 0 0 0) 0x7f3a6c0afa20 con 0x7f3a74103070 2024-09-10T19:09:56.746 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.743+0000 7f3a72d9d640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 0x7f3a5c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:56.747 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.743+0000 7f3a72d9d640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 0x7f3a5c071470 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004fd0 tx=0x7f3a64009040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:56.748 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.747+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f3a6c07bdb0 con 0x7f3a74103070 2024-09-10T19:09:56.904 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:56.903+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f3a7404bf60 con 0x7f3a74103070 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:56 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: pgmap v1155: 164 pgs: 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd crush rule rm", "name":"test-rados-api-smithi107-80290-111"}]': finished 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: osdmap e737: 8 total, 8 up, 8 in 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:56 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:57.551+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v739) v1 ==== 223+0+0 (secure 0 0 0) 0x7f3a6c041020 con 0x7f3a74103070 2024-09-10T19:09:57.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:57.655+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f3a74070ec0 con 0x7f3a74103070 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]': finished 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:57.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:57.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:57 smithi162 ceph-mon[33228]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T19:09:57.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]': finished 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[29404]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]': finished 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: osdmap e738: 8 total, 8 up, 8 in 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/4136161686' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pg_num": 12}]: dispatch 2024-09-10T19:09:57.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:57.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:57.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:57 smithi107 ceph-mon[40355]: osdmap e739: 8 total, 8 up, 8 in 2024-09-10T19:09:58.565 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.563+0000 7f3a5b7fe640 1 -- 172.21.15.107:0/260501348 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v740) v1 ==== 223+0+0 (secure 0 0 0) 0x7f3a6c07b750 con 0x7f3a74103070 2024-09-10T19:09:58.565 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 10 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:09:58.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.566+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 msgr2=0x7f3a5c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:58.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.566+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 0x7f3a5c071470 secure :-1 s=READY pgs=3594 cs=0 l=1 rev1=1 crypto rx=0x7f3a64004fd0 tx=0x7f3a64009040 comp rx=0 tx=0).stop 2024-09-10T19:09:58.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.566+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 msgr2=0x7f3a74070460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:58.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.566+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 0x7f3a74070460 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f3a6c02a840 tx=0x7f3a6c004d10 comp rx=0 tx=0).stop 2024-09-10T19:09:58.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 shutdown_connections 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f3a5c06efc0 0x7f3a5c071470 unknown :-1 s=CLOSED pgs=3594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f3a74103070 0x7f3a74070460 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f3a741026d0 0x7f3a7406ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 --2- 172.21.15.107:0/260501348 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f3a740fce70 0x7f3a7406f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 >> 172.21.15.107:0/260501348 conn(0x7f3a740f82e0 msgr2=0x7f3a740f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 shutdown_connections 2024-09-10T19:09:58.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.567+0000 7f3a79068640 1 -- 172.21.15.107:0/260501348 wait complete. 2024-09-10T19:09:58.579 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool application enable b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f rados 2024-09-10T19:09:58.663 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- 172.21.15.107:0/3585027934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80441015f0 msgr2=0x7f80441019c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:58.663 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 --2- 172.21.15.107:0/3585027934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80441015f0 0x7f80441019c0 secure :-1 s=READY pgs=2396 cs=0 l=1 rev1=1 crypto rx=0x7f8040004890 tx=0x7f8040009da0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.663 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- 172.21.15.107:0/3585027934 shutdown_connections 2024-09-10T19:09:58.663 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 --2- 172.21.15.107:0/3585027934 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f80440fbd30 0x7f80441083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 --2- 172.21.15.107:0/3585027934 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f8044101f90 0x7f80440fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 --2- 172.21.15.107:0/3585027934 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80441015f0 0x7f80441019c0 unknown :-1 s=CLOSED pgs=2396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- 172.21.15.107:0/3585027934 >> 172.21.15.107:0/3585027934 conn(0x7f80440725f0 msgr2=0x7f80440f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- 172.21.15.107:0/3585027934 shutdown_connections 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- 172.21.15.107:0/3585027934 wait complete. 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 Processor -- start 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.660+0000 7f804be6f640 1 -- start start 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80441015f0 0x7f804410e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8044101f90 0x7f804410e7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:58.664 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044118fb0 con 0x7f80440fbd30 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8044119120 con 0x7f8044101f90 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80441192e0 con 0x7f80441015f0 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:33260/0 (socket says 172.21.15.107:33260) 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804940a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80441015f0 0x7f804410e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 -- 172.21.15.107:0/106149436 learned_addr learned my addr 172.21.15.107:0/106149436 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804a40c640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8044101f90 0x7f804410e7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 -- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80441015f0 msgr2=0x7f804410e270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80441015f0 0x7f804410e270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 -- 172.21.15.107:0/106149436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8044101f90 msgr2=0x7f804410e7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8044101f90 0x7f804410e7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8034005040 con 0x7f80440fbd30 2024-09-10T19:09:58.665 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f8049c0b640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f80400048f0 tx=0x7f8040009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:58.666 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8040009350 con 0x7f80440fbd30 2024-09-10T19:09:58.666 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8040005040 con 0x7f80440fbd30 2024-09-10T19:09:58.666 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f80400094b0 con 0x7f80440fbd30 2024-09-10T19:09:58.666 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f804003b660 con 0x7f80440fbd30 2024-09-10T19:09:58.666 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.661+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80441198a0 con 0x7f80440fbd30 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.662+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f80440fcd70 con 0x7f80440fbd30 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.663+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f8040030020 con 0x7f80440fbd30 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.663+0000 7f803a7fc640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 0x7f8030071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.663+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(740..740 src has 1..740) v4 ==== 9055+0+0 (secure 0 0 0) 0x7f80400aef30 con 0x7f80440fbd30 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.663+0000 7f804940a640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 0x7f8030071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:09:58.667 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.664+0000 7f804940a640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 0x7f8030071470 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7f8044066450 tx=0x7f803400b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:09:58.669 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.667+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f804007b290 con 0x7f80440fbd30 2024-09-10T19:09:58.825 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:58.823+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"} v 0) v1 -- 0x7f804410a960 con 0x7f80440fbd30 2024-09-10T19:09:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:58 smithi162 ceph-mon[33228]: pgmap v1158: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:58 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:58 smithi107 ceph-mon[29404]: pgmap v1158: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:58 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:58.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:58 smithi107 ceph-mon[40355]: pgmap v1158: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:09:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:58 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:09:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:09:59] "GET /metrics HTTP/1.1" 200 228658 "" "Prometheus/2.43.0" 2024-09-10T19:09:59.577 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:59.576+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]=0 enabled application 'rados' on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' v741) v1 ==== 213+0+0 (secure 0 0 0) 0x7f804007ac30 con 0x7f80440fbd30 2024-09-10T19:09:59.673 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:09:59.671+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"} v 0) v1 -- 0x7f804404bf60 con 0x7f80440fbd30 2024-09-10T19:09:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:59 smithi162 ceph-mon[33228]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T19:09:59.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:09:59 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:09:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[40355]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T19:09:59.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:09:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/260501348' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:09:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[29404]: osdmap e740: 8 total, 8 up, 8 in 2024-09-10T19:09:59.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:09:59 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:10:00.584 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.583+0000 7f803a7fc640 1 -- 172.21.15.107:0/106149436 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]=0 enabled application 'rados' on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' v742) v1 ==== 213+0+0 (secure 0 0 0) 0x7f804007ae10 con 0x7f80440fbd30 2024-09-10T19:10:00.585 INFO:tasks.workunit.client.0.smithi107.stderr:enabled application 'rados' on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 msgr2=0x7f8030071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 0x7f8030071470 secure :-1 s=READY pgs=3595 cs=0 l=1 rev1=1 crypto rx=0x7f8044066450 tx=0x7f803400b040 comp rx=0 tx=0).stop 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 msgr2=0x7f804410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f80400048f0 tx=0x7f8040009da0 comp rx=0 tx=0).stop 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 shutdown_connections 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f803006efc0 0x7f8030071470 unknown :-1 s=CLOSED pgs=3595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f8044101f90 0x7f804410e7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:00.588 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f80441015f0 0x7f804410e270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:00.589 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 --2- 172.21.15.107:0/106149436 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f80440fbd30 0x7f804410dd30 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:00.589 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 >> 172.21.15.107:0/106149436 conn(0x7f80440725f0 msgr2=0x7f80440ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:10:00.589 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 shutdown_connections 2024-09-10T19:10:00.589 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:00.586+0000 7f804be6f640 1 -- 172.21.15.107:0/106149436 wait complete. 2024-09-10T19:10:00.599 INFO:tasks.workunit.client.0.smithi107.stderr:++ seq 1 10 2024-09-10T19:10:00.625 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.625 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj1 /etc/passwd 2024-09-10T19:10:00.656 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.656 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj2 /etc/passwd 2024-09-10T19:10:00.692 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.692 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj3 /etc/passwd 2024-09-10T19:10:00.724 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.724 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj4 /etc/passwd 2024-09-10T19:10:00.755 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.756 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj5 /etc/passwd 2024-09-10T19:10:00.786 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.786 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj6 /etc/passwd 2024-09-10T19:10:00.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.816 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj7 /etc/passwd 2024-09-10T19:10:00.846 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.846 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj8 /etc/passwd 2024-09-10T19:10:00.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.877 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj9 /etc/passwd 2024-09-10T19:10:00.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:10:00.907 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put obj10 /etc/passwd 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: pgmap v1161: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:10:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: application not enabled on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' 2024-09-10T19:10:00.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:00 smithi162 ceph-mon[33228]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:10:00.937 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-10T19:10:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: pgmap v1161: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T19:10:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: application not enabled on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[29404]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: pgmap v1161: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: osdmap e741: 8 total, 8 up, 8 in 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]: dispatch 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: Health detail: HEALTH_WARN 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: [WRN] POOL_APP_NOT_ENABLED: 2 pool(s) do not have an application enabled 2024-09-10T19:10:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: application not enabled on pool 'ceph_test_rados_api_asio' 2024-09-10T19:10:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: application not enabled on pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' 2024-09-10T19:10:00.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:00 smithi107 ceph-mon[40355]: use 'ceph osd pool application enable ', where is 'cephfs', 'rbd', 'rgw', or freeform for custom applications. 2024-09-10T19:10:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:01 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:01 smithi162 ceph-mon[33228]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T19:10:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[29404]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T19:10:01.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/106149436' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "app": "rados"}]': finished 2024-09-10T19:10:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[40355]: osdmap e742: 8 total, 8 up, 8 in 2024-09-10T19:10:01.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:02.922 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:02 smithi162 ceph-mon[33228]: pgmap v1164: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:10:02.922 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:02 smithi162 ceph-mon[33228]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:10:02.922 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:10:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a[66871]: logger=sqlstore.transactions t=2024-09-10T19:10:02.715155143Z level=info msg="Database locked, sleeping then retrying" error="database is locked" retry=0 code="database is locked" 2024-09-10T19:10:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:02 smithi107 ceph-mon[29404]: pgmap v1164: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:10:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:02 smithi107 ceph-mon[29404]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:10:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:02 smithi107 ceph-mon[40355]: pgmap v1164: 176 pgs: 12 creating+peering, 164 active+clean; 582 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:10:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:02 smithi107 ceph-mon[40355]: Health check update: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-09-10T19:10:03.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:04 smithi162 ceph-mon[33228]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:04 smithi162 ceph-mon[33228]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_objects: 10) 2024-09-10T19:10:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:04 smithi162 ceph-mon[33228]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:04 smithi162 ceph-mon[33228]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T19:10:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[29404]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[29404]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_objects: 10) 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[29404]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[29404]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[40355]: pgmap v1165: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 884 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[40355]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_objects: 10) 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[40355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:04 smithi107 ceph-mon[40355]: osdmap e743: 8 total, 8 up, 8 in 2024-09-10T19:10:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:06 smithi162 ceph-mon[33228]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:06.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:06 smithi107 ceph-mon[29404]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:06.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:06 smithi107 ceph-mon[40355]: pgmap v1167: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 22 KiB/s wr, 2 op/s 2024-09-10T19:10:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:08 smithi162 ceph-mon[33228]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T19:10:08.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:08 smithi107 ceph-mon[29404]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T19:10:09.041 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:08 smithi107 ceph-mon[40355]: pgmap v1168: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 17 KiB/s wr, 2 op/s 2024-09-10T19:10:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:09] "GET /metrics HTTP/1.1" 200 226209 "" "Prometheus/2.43.0" 2024-09-10T19:10:10.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:10 smithi162 ceph-mon[33228]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-10T19:10:10.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:10 smithi107 ceph-mon[29404]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-10T19:10:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:10 smithi107 ceph-mon[40355]: pgmap v1169: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 15 KiB/s wr, 2 op/s 2024-09-10T19:10:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:12 smithi162 ceph-mon[33228]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:10:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:12 smithi107 ceph-mon[29404]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:10:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:12 smithi107 ceph-mon[40355]: pgmap v1170: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:10:13.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:14 smithi162 ceph-mon[33228]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:14.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:14.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:14 smithi107 ceph-mon[29404]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:14 smithi107 ceph-mon[40355]: pgmap v1171: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:16 smithi162 ceph-mon[33228]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T19:10:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[29404]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T19:10:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[40355]: pgmap v1172: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 920 B/s rd, 0 op/s 2024-09-10T19:10:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:18 smithi162 ceph-mon[33228]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:18 smithi107 ceph-mon[29404]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:18 smithi107 ceph-mon[40355]: pgmap v1173: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:19] "GET /metrics HTTP/1.1" 200 226205 "" "Prometheus/2.43.0" 2024-09-10T19:10:20.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:20 smithi162 ceph-mon[33228]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:20.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:20 smithi107 ceph-mon[29404]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:20.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:20 smithi107 ceph-mon[40355]: pgmap v1174: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:22.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:22 smithi162 ceph-mon[33228]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:22.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:22 smithi107 ceph-mon[29404]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:22.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:22 smithi107 ceph-mon[40355]: pgmap v1175: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:23.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:24.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:24 smithi162 ceph-mon[33228]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:24.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:24 smithi107 ceph-mon[40355]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:24.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:24 smithi107 ceph-mon[29404]: pgmap v1176: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:26.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:26 smithi162 ceph-mon[33228]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:26.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:26 smithi107 ceph-mon[40355]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:26.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:26 smithi107 ceph-mon[29404]: pgmap v1177: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:28.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:28 smithi162 ceph-mon[33228]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:28.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:28 smithi107 ceph-mon[29404]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:28.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:28 smithi107 ceph-mon[40355]: pgmap v1178: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:29] "GET /metrics HTTP/1.1" 200 226205 "" "Prometheus/2.43.0" 2024-09-10T19:10:30.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:30 smithi162 ceph-mon[33228]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:30.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:30 smithi107 ceph-mon[29404]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:30.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:30 smithi107 ceph-mon[40355]: pgmap v1179: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:30.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=149101 2024-09-10T19:10:30.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_objects 100 2024-09-10T19:10:30.939 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put onemore /etc/passwd 2024-09-10T19:10:31.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/3529099974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 msgr2=0x7f5cec101af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:31.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/3529099974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec101af0 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8004950 tx=0x7f5ce8009da0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/3529099974 shutdown_connections 2024-09-10T19:10:31.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/3529099974 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec0fbf90 0x7f5cec108520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/3529099974 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec102030 0x7f5cec0fb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/3529099974 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec101af0 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/3529099974 >> 172.21.15.107:0/3529099974 conn(0x7f5cec072600 msgr2=0x7f5cec072a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/3529099974 shutdown_connections 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.026+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/3529099974 wait complete. 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 Processor -- start 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 -- start start 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec102030 0x7f5cec065cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec066340 con 0x7f5cec101720 2024-09-10T19:10:31.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec0664b0 con 0x7f5cec102030 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf2cc9640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cec066670 con 0x7f5cec0fbf90 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5ce3fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:49100/0 (socket says 172.21.15.107:49100) 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 -- 172.21.15.107:0/2913491420 learned_addr learned my addr 172.21.15.107:0/2913491420 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf1266640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec102030 0x7f5cec065cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 -- 172.21.15.107:0/2913491420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec102030 msgr2=0x7f5cec065cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec102030 0x7f5cec065cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 -- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 msgr2=0x7f5cec113030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec113030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5cf0a65640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cdc005040 con 0x7f5cec0fbf90 2024-09-10T19:10:31.032 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.027+0000 7f5ce3fff640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5cf0a65640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009040 tx=0x7f5ce8009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce80092e0 con 0x7f5cec0fbf90 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f5ce803c880 con 0x7f5cec0fbf90 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ce8005040 con 0x7f5cec0fbf90 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ce8033650 con 0x7f5cec0fbf90 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.028+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cec119940 con 0x7f5cec0fbf90 2024-09-10T19:10:31.033 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.029+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cec04bf60 con 0x7f5cec0fbf90 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.029+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f5ce8030020 con 0x7f5cec0fbf90 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.030+0000 7f5ce17fa640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 0x7f5cd4071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.030+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(743..743 src has 1..743) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f5ce80aecf0 con 0x7f5cec0fbf90 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.030+0000 7f5ce3fff640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 0x7f5cd4071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.030+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=744}) v3 -- 0x7f5cd407a7c0 con 0x7f5cec0fbf90 2024-09-10T19:10:31.034 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.030+0000 7f5ce3fff640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 0x7f5cd4071470 secure :-1 s=READY pgs=3607 cs=0 l=1 rev1=1 crypto rx=0x7f5cec10e3b0 tx=0x7f5cdc00b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:10:31.035 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.034+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f5ce80b3070 con 0x7f5cec0fbf90 2024-09-10T19:10:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:31.193 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.192+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f5cec10e1f0 con 0x7f5cec0fbf90 2024-09-10T19:10:31.417 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.416+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 7 ==== osd_map(744..744 src has 1..744) v4 ==== 620+0+0 (secure 0 0 0) 0x7f5ce807ae40 con 0x7f5cec0fbf90 2024-09-10T19:10:31.418 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.416+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=745}) v3 -- 0x7f5cd407b350 con 0x7f5cec0fbf90 2024-09-10T19:10:31.421 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.419+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v744) v1 ==== 225+0+0 (secure 0 0 0) 0x7f5ce807ba30 con 0x7f5cec0fbf90 2024-09-10T19:10:31.515 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:31.513+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"} v 0) v1 -- 0x7f5cec0fcdc0 con 0x7f5cec0fbf90 2024-09-10T19:10:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:31 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:31.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:31 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:31.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:31.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:31 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.419 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.417+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 9 ==== osd_map(745..745 src has 1..745) v4 ==== 620+0+0 (secure 0 0 0) 0x7f5ce807ba30 con 0x7f5cec0fbf90 2024-09-10T19:10:32.419 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.417+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=746}) v3 -- 0x7f5cd407b820 con 0x7f5cec0fbf90 2024-09-10T19:10:32.423 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.421+0000 7f5ce17fa640 1 -- 172.21.15.107:0/2913491420 <== mon.2 v2:172.21.15.107:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]=0 set-quota max_objects = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v745) v1 ==== 225+0+0 (secure 0 0 0) 0x7f5ce80824c0 con 0x7f5cec0fbf90 2024-09-10T19:10:32.423 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:10:32.427 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.425+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 msgr2=0x7f5cd4071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:32.427 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.425+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 0x7f5cd4071470 secure :-1 s=READY pgs=3607 cs=0 l=1 rev1=1 crypto rx=0x7f5cec10e3b0 tx=0x7f5cdc00b040 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.425+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 msgr2=0x7f5cec112af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.425+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 secure :-1 s=READY pgs=2750 cs=0 l=1 rev1=1 crypto rx=0x7f5ce8009040 tx=0x7f5ce8009da0 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.425+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 shutdown_connections 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f5cd406efc0 0x7f5cd4071470 unknown :-1 s=CLOSED pgs=3607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f5cec102030 0x7f5cec065cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f5cec101720 0x7f5cec113030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 --2- 172.21.15.107:0/2913491420 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f5cec0fbf90 0x7f5cec112af0 unknown :-1 s=CLOSED pgs=2750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 >> 172.21.15.107:0/2913491420 conn(0x7f5cec072600 msgr2=0x7f5cec0ff2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 shutdown_connections 2024-09-10T19:10:32.428 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:32.426+0000 7f5cf2cc9640 1 -- 172.21.15.107:0/2913491420 wait complete. 2024-09-10T19:10:32.438 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 149101 2024-09-10T19:10:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[29404]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:32.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[29404]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[40355]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[40355]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T19:10:32.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:32 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-mon[33228]: pgmap v1180: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-mon[33228]: osdmap e744: 8 total, 8 up, 8 in 2024-09-10T19:10:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2913491420' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]: dispatch 2024-09-10T19:10:33.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:33 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:33.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:33 smithi107 ceph-mon[29404]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T19:10:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:33 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:33.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:33 smithi107 ceph-mon[40355]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T19:10:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:33 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "100"}]': finished 2024-09-10T19:10:33.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:33 smithi162 ceph-mon[33228]: osdmap e745: 8 total, 8 up, 8 in 2024-09-10T19:10:34.231 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 0 -ne 0 ']' 2024-09-10T19:10:34.231 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-10T19:10:34.231 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put twomore /etc/passwd 2024-09-10T19:10:34.264 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_bytes 100 2024-09-10T19:10:34.347 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.343+0000 7f9cc8617640 1 -- 172.21.15.107:0/4103094143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc00fbd30 msgr2=0x7f9cc01083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:34.347 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.343+0000 7f9cc8617640 1 --2- 172.21.15.107:0/4103094143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc00fbd30 0x7f9cc01083d0 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f9cb8004950 tx=0x7f9cb8009da0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.347 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- 172.21.15.107:0/4103094143 shutdown_connections 2024-09-10T19:10:34.347 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- 172.21.15.107:0/4103094143 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc00fbd30 0x7f9cc01083d0 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.347 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- 172.21.15.107:0/4103094143 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 0x7f9cc00fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- 172.21.15.107:0/4103094143 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc01015f0 0x7f9cc01019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- 172.21.15.107:0/4103094143 >> 172.21.15.107:0/4103094143 conn(0x7f9cc0072530 msgr2=0x7f9cc00f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- 172.21.15.107:0/4103094143 shutdown_connections 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- 172.21.15.107:0/4103094143 wait complete. 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 Processor -- start 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- start start 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 0x7f9cc0110ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 0x7f9cc010b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010be50 con 0x7f9cc01015f0 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010bfc0 con 0x7f9cc0101f90 2024-09-10T19:10:34.348 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc8617640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc010c180 con 0x7f9cc00fbd30 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc63b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc5bb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 0x7f9cc0110ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.344+0000 7f9cc63b3640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:37488/0 (socket says 172.21.15.107:37488) 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc5bb2640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 0x7f9cc0110ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:50584/0 (socket says 172.21.15.107:50584) 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 -- 172.21.15.107:0/3926423848 learned_addr learned my addr 172.21.15.107:0/3926423848 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc6bb4640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 0x7f9cc010b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 -- 172.21.15.107:0/3926423848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 msgr2=0x7f9cc010b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 0x7f9cc010b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.349 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 -- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 msgr2=0x7f9cc0110ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 0x7f9cc0110ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb00098b0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc63b3640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0001f00 tx=0x7f9cb0004c50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb000fea0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f9cb0007cd0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cb0009700 con 0x7f9cc00fbd30 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cb8005040 con 0x7f9cc00fbd30 2024-09-10T19:10:34.350 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.345+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc01184f0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.346+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc00fcd70 con 0x7f9cc00fbd30 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.347+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f9cb0007890 con 0x7f9cc00fbd30 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.347+0000 7f9cbeffd640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 0x7f9ca8071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.348+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(746..746 src has 1..746) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f9cb008f4b0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.348+0000 7f9cc5bb2640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 0x7f9ca8071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:10:34.351 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.348+0000 7f9cc5bb2640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 0x7f9ca8071470 secure :-1 s=READY pgs=3609 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00fb9a0 tx=0x7f9cb4004330 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:10:34.353 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.352+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f9cb004f2a0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.512 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:34.510+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f9cc004bfe0 con 0x7f9cc00fbd30 2024-09-10T19:10:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[29404]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:34.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[29404]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[29404]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[29404]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[40355]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[40355]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[40355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:10:34.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:34 smithi107 ceph-mon[40355]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T19:10:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:34 smithi162 ceph-mon[33228]: pgmap v1183: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:34 smithi162 ceph-mon[33228]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:10:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:34 smithi162 ceph-mon[33228]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:10:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:34 smithi162 ceph-mon[33228]: osdmap e746: 8 total, 8 up, 8 in 2024-09-10T19:10:35.441 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:35.439+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v747) v1 ==== 221+0+0 (secure 0 0 0) 0x7f9cb005b830 con 0x7f9cc00fbd30 2024-09-10T19:10:35.537 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:35.535+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"} v 0) v1 -- 0x7f9cc010cce0 con 0x7f9cc00fbd30 2024-09-10T19:10:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:35 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:35.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:35 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:35 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:35.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:35 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:35 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:35.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:35 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.443 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.441+0000 7f9cbeffd640 1 -- 172.21.15.107:0/3926423848 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]=0 set-quota max_bytes = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v748) v1 ==== 221+0+0 (secure 0 0 0) 0x7f9cb001a070 con 0x7f9cc00fbd30 2024-09-10T19:10:36.443 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 100 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:10:36.446 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.444+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 msgr2=0x7f9ca8071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:36.446 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.444+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 0x7f9ca8071470 secure :-1 s=READY pgs=3609 cs=0 l=1 rev1=1 crypto rx=0x7f9cc00fb9a0 tx=0x7f9cb4004330 comp rx=0 tx=0).stop 2024-09-10T19:10:36.446 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.444+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 msgr2=0x7f9cc01105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:10:36.446 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.444+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 secure :-1 s=READY pgs=2752 cs=0 l=1 rev1=1 crypto rx=0x7f9cb0001f00 tx=0x7f9cb0004c50 comp rx=0 tx=0).stop 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 shutdown_connections 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f9ca806efc0 0x7f9ca8071470 unknown :-1 s=CLOSED pgs=3609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f9cc0101f90 0x7f9cc010b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f9cc01015f0 0x7f9cc0110ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 --2- 172.21.15.107:0/3926423848 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f9cc00fbd30 0x7f9cc01105a0 unknown :-1 s=CLOSED pgs=2752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 >> 172.21.15.107:0/3926423848 conn(0x7f9cc0072530 msgr2=0x7f9cc0073250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 shutdown_connections 2024-09-10T19:10:36.447 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:10:36.445+0000 7f9cc8617640 1 -- 172.21.15.107:0/3926423848 wait complete. 2024-09-10T19:10:36.457 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[29404]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[29404]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[40355]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:36.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:36.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[40355]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T19:10:36.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.717 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:36 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:36 smithi162 ceph-mon[33228]: pgmap v1185: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:36 smithi162 ceph-mon[33228]: osdmap e747: 8 total, 8 up, 8 in 2024-09-10T19:10:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:36 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3926423848' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:36 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]: dispatch 2024-09-10T19:10:37.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:37 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:37.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:37 smithi107 ceph-mon[29404]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T19:10:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:37 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:37.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:37 smithi107 ceph-mon[40355]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T19:10:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:37 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "100"}]': finished 2024-09-10T19:10:37.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:37 smithi162 ceph-mon[33228]: osdmap e748: 8 total, 8 up, 8 in 2024-09-10T19:10:38.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:38 smithi107 ceph-mon[29404]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T19:10:38.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:38 smithi107 ceph-mon[40355]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T19:10:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:38 smithi162 ceph-mon[33228]: pgmap v1188: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 5.3 KiB/s wr, 1 op/s 2024-09-10T19:10:39.451 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:39] "GET /metrics HTTP/1.1" 200 226171 "" "Prometheus/2.43.0" 2024-09-10T19:10:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[29404]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_bytes: 100 B) 2024-09-10T19:10:39.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[29404]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:39.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[29404]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T19:10:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[40355]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_bytes: 100 B) 2024-09-10T19:10:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[40355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:39.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:39 smithi107 ceph-mon[40355]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T19:10:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:39 smithi162 ceph-mon[33228]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' is full (reached quota's max_bytes: 100 B) 2024-09-10T19:10:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:39 smithi162 ceph-mon[33228]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:10:39.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:39 smithi162 ceph-mon[33228]: osdmap e749: 8 total, 8 up, 8 in 2024-09-10T19:10:40.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:40 smithi107 ceph-mon[29404]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T19:10:40.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:40 smithi107 ceph-mon[40355]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T19:10:40.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:40 smithi162 ceph-mon[33228]: pgmap v1190: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 998 B/s rd, 5.1 KiB/s wr, 1 op/s 2024-09-10T19:10:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:42.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:42 smithi107 ceph-mon[29404]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T19:10:42.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:42 smithi107 ceph-mon[40355]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T19:10:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:42 smithi162 ceph-mon[33228]: pgmap v1191: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 4.3 KiB/s wr, 1 op/s 2024-09-10T19:10:43.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:44.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:44.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:44 smithi107 ceph-mon[29404]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T19:10:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:44.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:44 smithi107 ceph-mon[40355]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T19:10:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:44 smithi162 ceph-mon[33228]: pgmap v1192: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 3.3 KiB/s wr, 1 op/s 2024-09-10T19:10:46.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:46 smithi107 ceph-mon[29404]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T19:10:46.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:46 smithi107 ceph-mon[40355]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T19:10:46.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:46 smithi162 ceph-mon[33228]: pgmap v1193: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 2.9 KiB/s wr, 1 op/s 2024-09-10T19:10:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:10:48.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:48 smithi107 ceph-mon[29404]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:48.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:48 smithi107 ceph-mon[40355]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:48 smithi162 ceph-mon[33228]: pgmap v1194: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:10:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:49] "GET /metrics HTTP/1.1" 200 226188 "" "Prometheus/2.43.0" 2024-09-10T19:10:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-mon[33228]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T19:10:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:10:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:10:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:10:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[29404]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T19:10:50.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[40355]: pgmap v1195: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1010 B/s rd, 0 op/s 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:10:50.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:10:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:10:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:10:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:52 smithi162 ceph-mon[33228]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:52 smithi107 ceph-mon[29404]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:52.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:52 smithi107 ceph-mon[40355]: pgmap v1196: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:53.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:10:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:10:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:54 smithi162 ceph-mon[33228]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:54 smithi107 ceph-mon[29404]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:10:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:54 smithi107 ceph-mon[40355]: pgmap v1197: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:56 smithi162 ceph-mon[33228]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:56 smithi107 ceph-mon[29404]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:56.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:56 smithi107 ceph-mon[40355]: pgmap v1198: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:10:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:10:58 smithi162 ceph-mon[33228]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:10:58 smithi107 ceph-mon[29404]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:10:58 smithi107 ceph-mon[40355]: pgmap v1199: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:10:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:10:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:10:59] "GET /metrics HTTP/1.1" 200 226188 "" "Prometheus/2.43.0" 2024-09-10T19:11:00.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:00 smithi162 ceph-mon[33228]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:00 smithi107 ceph-mon[29404]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:00 smithi107 ceph-mon[40355]: pgmap v1200: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:01.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:01.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:01 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:01.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:01 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:01.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:01 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:02.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:02 smithi162 ceph-mon[33228]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:02.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:02 smithi107 ceph-mon[29404]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:02.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:02 smithi107 ceph-mon[40355]: pgmap v1201: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:03.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:02 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:04 smithi162 ceph-mon[33228]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:04.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:04 smithi107 ceph-mon[29404]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:04.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:04 smithi107 ceph-mon[40355]: pgmap v1202: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:06.459 INFO:tasks.workunit.client.0.smithi107.stderr:+ pid=149320 2024-09-10T19:11:06.459 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_bytes 0 2024-09-10T19:11:06.459 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put two /etc/passwd 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 -- 172.21.15.107:0/1443653788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8380fce70 msgr2=0x7fe838109590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 --2- 172.21.15.107:0/1443653788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8380fce70 0x7fe838109590 secure :-1 s=READY pgs=2754 cs=0 l=1 rev1=1 crypto rx=0x7fe830009d00 tx=0x7fe830004d10 comp rx=0 tx=0).stop 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 -- 172.21.15.107:0/1443653788 shutdown_connections 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 --2- 172.21.15.107:0/1443653788 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe8380fce70 0x7fe838109590 unknown :-1 s=CLOSED pgs=2754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 --2- 172.21.15.107:0/1443653788 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe838103070 0x7fe8380fc740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 --2- 172.21.15.107:0/1443653788 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 0x7fe838102aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:06.552 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 -- 172.21.15.107:0/1443653788 >> 172.21.15.107:0/1443653788 conn(0x7fe8380f82e0 msgr2=0x7fe8380fa6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 -- 172.21.15.107:0/1443653788 shutdown_connections 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.547+0000 7fe83c99d640 1 -- 172.21.15.107:0/1443653788 wait complete. 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 Processor -- start 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 -- start start 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 0x7fe838111c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 0x7fe83810c990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83810cf90 con 0x7fe8380fce70 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe83810d100 con 0x7fe8381026d0 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83c99d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe83810d2c0 con 0x7fe838103070 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:06.553 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48676/0 (socket says 172.21.15.107:48676) 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe836d9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 0x7fe83810c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 -- 172.21.15.107:0/2284422189 learned_addr learned my addr 172.21.15.107:0/2284422189 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe835d9b640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 0x7fe838111c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 -- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 msgr2=0x7fe83810c990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 0x7fe83810c990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 -- 172.21.15.107:0/2284422189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 msgr2=0x7fe838111c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 0x7fe838111c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe82c005040 con 0x7fe8380fce70 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe836d9d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 0x7fe83810c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.548+0000 7fe83659c640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe828002ad0 tx=0x7fe828004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.549+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828007c30 con 0x7fe8380fce70 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.549+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fe82800f040 con 0x7fe8380fce70 2024-09-10T19:11:06.554 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.549+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8300098b0 con 0x7fe8380fce70 2024-09-10T19:11:06.555 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.549+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe828009340 con 0x7fe8380fce70 2024-09-10T19:11:06.555 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.549+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe838119630 con 0x7fe8380fce70 2024-09-10T19:11:06.555 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.550+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8380fde80 con 0x7fe8380fce70 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.550+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fe828009770 con 0x7fe8380fce70 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.551+0000 7fe826ffd640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 0x7fe820071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.551+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(749..749 src has 1..749) v4 ==== 9068+0+0 (secure 0 0 0) 0x7fe82808ed70 con 0x7fe8380fce70 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.551+0000 7fe835d9b640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 0x7fe820071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.551+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=750}) v3 -- 0x7fe82007a7c0 con 0x7fe8380fce70 2024-09-10T19:11:06.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.551+0000 7fe835d9b640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 0x7fe820071470 secure :-1 s=READY pgs=3611 cs=0 l=1 rev1=1 crypto rx=0x7fe83810dd80 tx=0x7fe82c00b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:06.557 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.555+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fe82805b0f0 con 0x7fe8380fce70 2024-09-10T19:11:06.713 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:06.710+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe83810b960 con 0x7fe8380fce70 2024-09-10T19:11:06.767 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:06 smithi107 ceph-mon[29404]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:06.767 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:06 smithi107 ceph-mon[40355]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:06 smithi162 ceph-mon[33228]: pgmap v1203: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:07.512 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:07.509+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 7 ==== osd_map(750..750 src has 1..750) v4 ==== 620+0+0 (secure 0 0 0) 0x7fe828092030 con 0x7fe8380fce70 2024-09-10T19:11:07.512 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:07.509+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=751}) v3 -- 0x7fe82007b390 con 0x7fe8380fce70 2024-09-10T19:11:07.519 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:07.517+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v750) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe82805aa90 con 0x7fe8380fce70 2024-09-10T19:11:07.614 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:07.611+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7fe83804bf60 con 0x7fe8380fce70 2024-09-10T19:11:07.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:07 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:07.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:07 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:07.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:07 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:08.516 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.513+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 9 ==== osd_map(751..751 src has 1..751) v4 ==== 620+0+0 (secure 0 0 0) 0x7fe828007e20 con 0x7fe8380fce70 2024-09-10T19:11:08.516 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.513+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=752}) v3 -- 0x7fe82007b860 con 0x7fe8380fce70 2024-09-10T19:11:08.520 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.518+0000 7fe826ffd640 1 -- 172.21.15.107:0/2284422189 <== mon.0 v2:172.21.15.107:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v751) v1 ==== 217+0+0 (secure 0 0 0) 0x7fe82805ac70 con 0x7fe8380fce70 2024-09-10T19:11:08.521 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:11:08.524 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.521+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 msgr2=0x7fe820071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:08.524 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.521+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 0x7fe820071470 secure :-1 s=READY pgs=3611 cs=0 l=1 rev1=1 crypto rx=0x7fe83810dd80 tx=0x7fe82c00b040 comp rx=0 tx=0).stop 2024-09-10T19:11:08.524 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.521+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 msgr2=0x7fe8381116e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:08.524 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.521+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7fe828002ad0 tx=0x7fe828004600 comp rx=0 tx=0).stop 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 shutdown_connections 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fe82006efc0 0x7fe820071470 unknown :-1 s=CLOSED pgs=3611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fe838103070 0x7fe83810c990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fe8381026d0 0x7fe838111c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 --2- 172.21.15.107:0/2284422189 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fe8380fce70 0x7fe8381116e0 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 >> 172.21.15.107:0/2284422189 conn(0x7fe8380f82e0 msgr2=0x7fe8380f97d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 shutdown_connections 2024-09-10T19:11:08.525 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.522+0000 7fe83c99d640 1 -- 172.21.15.107:0/2284422189 wait complete. 2024-09-10T19:11:08.535 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_objects 0 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.614+0000 7f6e3079d640 1 -- 172.21.15.107:0/3729831194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fd6f0 msgr2=0x7f6e280fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- 172.21.15.107:0/3729831194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fd6f0 0x7f6e280fdac0 secure :-1 s=READY pgs=2755 cs=0 l=1 rev1=1 crypto rx=0x7f6e24004890 tx=0x7f6e24009da0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- 172.21.15.107:0/3729831194 shutdown_connections 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- 172.21.15.107:0/3729831194 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e28109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- 172.21.15.107:0/3729831194 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fe090 0x7f6e28106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- 172.21.15.107:0/3729831194 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fd6f0 0x7f6e280fdac0 unknown :-1 s=CLOSED pgs=2755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- 172.21.15.107:0/3729831194 >> 172.21.15.107:0/3729831194 conn(0x7f6e28072530 msgr2=0x7f6e280f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- 172.21.15.107:0/3729831194 shutdown_connections 2024-09-10T19:11:08.620 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- 172.21.15.107:0/3729831194 wait complete. 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 Processor -- start 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- start start 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fe090 0x7f6e281130c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e2810cd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2810d3b0 con 0x7f6e280fd6f0 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2810d520 con 0x7f6e28106ca0 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e3079d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e2810d6e0 con 0x7f6e280fe090 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e2e539640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e2e539640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48690/0 (socket says 172.21.15.107:48690) 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.615+0000 7f6e2e539640 1 -- 172.21.15.107:0/17591372 learned_addr learned my addr 172.21.15.107:0/17591372 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:11:08.621 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2dd38640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fe090 0x7f6e281130c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2ed3a640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e2810cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 -- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fe090 msgr2=0x7f6e281130c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fe090 0x7f6e281130c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 -- 172.21.15.107:0/17591372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 msgr2=0x7f6e2810cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e2810cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e18005040 con 0x7f6e280fd6f0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2ed3a640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e2810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e2e539640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009040 tx=0x7f6e24035040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e240093c0 con 0x7f6e280fd6f0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f6e24009520 con 0x7f6e280fd6f0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e24005040 con 0x7f6e280fd6f0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e24033420 con 0x7f6e280fd6f0 2024-09-10T19:11:08.622 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.616+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e281199d0 con 0x7f6e280fd6f0 2024-09-10T19:11:08.623 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.617+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e28065310 con 0x7f6e280fd6f0 2024-09-10T19:11:08.623 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f6e24030020 con 0x7f6e280fd6f0 2024-09-10T19:11:08.623 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e1effd640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 0x7f6e14071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:08.623 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(751..751 src has 1..751) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f6e240aede0 con 0x7f6e280fd6f0 2024-09-10T19:11:08.624 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e2dd38640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 0x7f6e14071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:08.624 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=752}) v3 -- 0x7f6e1407a840 con 0x7f6e280fd6f0 2024-09-10T19:11:08.624 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.618+0000 7f6e2dd38640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 0x7f6e14071470 secure :-1 s=READY pgs=3612 cs=0 l=1 rev1=1 crypto rx=0x7f6e18007fd0 tx=0x7f6e1800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:08.624 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.622+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f6e2407b270 con 0x7f6e280fd6f0 2024-09-10T19:11:08.779 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:08.776+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f6e2810b980 con 0x7f6e280fd6f0 2024-09-10T19:11:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:08 smithi162 ceph-mon[33228]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:08 smithi162 ceph-mon[33228]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T19:11:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:08 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[29404]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[29404]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[40355]: pgmap v1204: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[40355]: osdmap e750: 8 total, 8 up, 8 in 2024-09-10T19:11:08.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:08 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:11:09.228 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:09.225+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 7 ==== osd_map(752..752 src has 1..752) v4 ==== 620+0+0 (secure 0 0 0) 0x7f6e2407afb0 con 0x7f6e280fd6f0 2024-09-10T19:11:09.228 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:09.226+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=753}) v3 -- 0x7f6e1407b3d0 con 0x7f6e280fd6f0 2024-09-10T19:11:09.233 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:09.229+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v752) v1 ==== 221+0+0 (secure 0 0 0) 0x7f6e2407ac10 con 0x7f6e280fd6f0 2024-09-10T19:11:09.329 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:09.327+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7f6e2810e1f0 con 0x7f6e280fd6f0 2024-09-10T19:11:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:09] "GET /metrics HTTP/1.1" 200 226165 "" "Prometheus/2.43.0" 2024-09-10T19:11:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T19:11:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:09.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:11:09.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:11:09.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:09.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T19:11:09.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:09 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T19:11:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:09.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2284422189' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: osdmap e751: 8 total, 8 up, 8 in 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' no longer out of quota; removing NO_QUOTA flag 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: osdmap e752: 8 total, 8 up, 8 in 2024-09-10T19:11:09.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:09 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:11:10.234 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.231+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 9 ==== osd_map(753..753 src has 1..753) v4 ==== 620+0+0 (secure 0 0 0) 0x7f6e2407bba0 con 0x7f6e280fd6f0 2024-09-10T19:11:10.238 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.235+0000 7f6e1effd640 1 -- 172.21.15.107:0/17591372 <== mon.0 v2:172.21.15.107:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v753) v1 ==== 221+0+0 (secure 0 0 0) 0x7f6e2407adf0 con 0x7f6e280fd6f0 2024-09-10T19:11:10.238 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 msgr2=0x7f6e14071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 0x7f6e14071470 secure :-1 s=READY pgs=3612 cs=0 l=1 rev1=1 crypto rx=0x7f6e18007fd0 tx=0x7f6e1800b040 comp rx=0 tx=0).stop 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 msgr2=0x7f6e28112b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 secure :-1 s=READY pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009040 tx=0x7f6e24035040 comp rx=0 tx=0).stop 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 shutdown_connections 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f6e1406efc0 0x7f6e14071470 unknown :-1 s=CLOSED pgs=3612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f6e28106ca0 0x7f6e2810cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f6e280fe090 0x7f6e281130c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.242 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 --2- 172.21.15.107:0/17591372 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f6e280fd6f0 0x7f6e28112b80 secure :-1 s=CLOSED pgs=2421 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009040 tx=0x7f6e24035040 comp rx=0 tx=0).stop 2024-09-10T19:11:10.243 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 >> 172.21.15.107:0/17591372 conn(0x7f6e28072530 msgr2=0x7f6e280fe6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:10.243 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 shutdown_connections 2024-09-10T19:11:10.243 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.239+0000 7f6e3079d640 1 -- 172.21.15.107:0/17591372 wait complete. 2024-09-10T19:11:10.253 INFO:tasks.workunit.client.0.smithi107.stderr:+ wait 149320 2024-09-10T19:11:10.253 INFO:tasks.workunit.client.0.smithi107.stderr:+ '[' 0 -ne 0 ']' 2024-09-10T19:11:10.253 INFO:tasks.workunit.client.0.smithi107.stderr:+ true 2024-09-10T19:11:10.253 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put three /etc/passwd 2024-09-10T19:11:10.286 INFO:tasks.workunit.client.0.smithi107.stderr:++ uuidgen 2024-09-10T19:11:10.286 INFO:tasks.workunit.client.0.smithi107.stderr:+ pp=6328337b-b283-4924-8680-a2da095e48b3 2024-09-10T19:11:10.286 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool create 6328337b-b283-4924-8680-a2da095e48b3 12 2024-09-10T19:11:10.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 -- 172.21.15.107:0/1264230491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c0fbd30 msgr2=0x7fd44c1083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:10.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 --2- 172.21.15.107:0/1264230491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c0fbd30 0x7fd44c1083d0 secure :-1 s=READY pgs=2756 cs=0 l=1 rev1=1 crypto rx=0x7fd444009cd0 tx=0x7fd444004d10 comp rx=0 tx=0).stop 2024-09-10T19:11:10.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 -- 172.21.15.107:0/1264230491 shutdown_connections 2024-09-10T19:11:10.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 --2- 172.21.15.107:0/1264230491 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c0fbd30 0x7fd44c1083d0 unknown :-1 s=CLOSED pgs=2756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.368 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 --2- 172.21.15.107:0/1264230491 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 0x7fd44c0fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 --2- 172.21.15.107:0/1264230491 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c1015f0 0x7fd44c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 -- 172.21.15.107:0/1264230491 >> 172.21.15.107:0/1264230491 conn(0x7fd44c072510 msgr2=0x7fd44c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 -- 172.21.15.107:0/1264230491 shutdown_connections 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.363+0000 7fd45435d640 1 -- 172.21.15.107:0/1264230491 wait complete. 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 Processor -- start 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 -- start start 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 0x7fd44c112cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 0x7fd44c10c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10cf60 con 0x7fd44c0fbd30 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10d0d0 con 0x7fd44c101f90 2024-09-10T19:11:10.369 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd45435d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd44c10d290 con 0x7fd44c1015f0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4520f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 0x7fd44c112cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4520f9640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48722/0 (socket says 172.21.15.107:48722) 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4520f9640 1 -- 172.21.15.107:0/2558947535 learned_addr learned my addr 172.21.15.107:0/2558947535 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 -- 172.21.15.107:0/2558947535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 msgr2=0x7fd44c10c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4528fa640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 0x7fd44c10c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 0x7fd44c10c960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 -- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 msgr2=0x7fd44c112770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4518f8640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd448005040 con 0x7fd44c1015f0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.364+0000 7fd4520f9640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd4518f8640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 0x7fd44c112cb0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fd44c065a80 tx=0x7fd43c007f20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:10.370 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c00a610 con 0x7fd44c1015f0 2024-09-10T19:11:10.371 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4440098b0 con 0x7fd44c1015f0 2024-09-10T19:11:10.371 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fd43c00abb0 con 0x7fd44c1015f0 2024-09-10T19:11:10.371 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43c012a90 con 0x7fd44c1015f0 2024-09-10T19:11:10.371 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.365+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd44c119630 con 0x7fd44c1015f0 2024-09-10T19:11:10.371 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.366+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd44c0fcd70 con 0x7fd44c1015f0 2024-09-10T19:11:10.372 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.366+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fd43c00a770 con 0x7fd44c1015f0 2024-09-10T19:11:10.372 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.367+0000 7fd442ffd640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 0x7fd438071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:10.372 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.367+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(753..753 src has 1..753) v4 ==== 9068+0+0 (secure 0 0 0) 0x7fd43c08ef40 con 0x7fd44c1015f0 2024-09-10T19:11:10.372 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.367+0000 7fd4520f9640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 0x7fd438071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:10.372 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.367+0000 7fd4520f9640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 0x7fd438071470 secure :-1 s=READY pgs=3614 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d70 tx=0x7fd44800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:10.373 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.371+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fd43c05b290 con 0x7fd44c1015f0 2024-09-10T19:11:10.531 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:10.528+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12} v 0) v1 -- 0x7fd44c10a840 con 0x7fd44c1015f0 2024-09-10T19:11:10.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:10 smithi162 ceph-mon[33228]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:10.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:10 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:10.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:10 smithi162 ceph-mon[33228]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T19:11:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[29404]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:10.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[29404]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T19:11:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[40355]: pgmap v1208: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/17591372' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:11:10.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:10 smithi107 ceph-mon[40355]: osdmap e753: 8 total, 8 up, 8 in 2024-09-10T19:11:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:11.542 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.540+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]=0 pool '6328337b-b283-4924-8680-a2da095e48b3' created v754) v1 ==== 176+0+0 (secure 0 0 0) 0x7fd43c05ac30 con 0x7fd44c1015f0 2024-09-10T19:11:11.638 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.637+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12} v 0) v1 -- 0x7fd44c04bf60 con 0x7fd44c1015f0 2024-09-10T19:11:11.639 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.637+0000 7fd442ffd640 1 -- 172.21.15.107:0/2558947535 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]=0 pool '6328337b-b283-4924-8680-a2da095e48b3' already exists v754) v1 ==== 183+0+0 (secure 0 0 0) 0x7fd43c05ac30 con 0x7fd44c1015f0 2024-09-10T19:11:11.639 INFO:tasks.workunit.client.0.smithi107.stderr:pool '6328337b-b283-4924-8680-a2da095e48b3' already exists 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 msgr2=0x7fd438071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 0x7fd438071470 secure :-1 s=READY pgs=3614 cs=0 l=1 rev1=1 crypto rx=0x7fd448009d70 tx=0x7fd44800b040 comp rx=0 tx=0).stop 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 msgr2=0x7fd44c112cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 0x7fd44c112cb0 secure :-1 s=READY pgs=2757 cs=0 l=1 rev1=1 crypto rx=0x7fd44c065a80 tx=0x7fd43c007f20 comp rx=0 tx=0).stop 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 shutdown_connections 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fd43806efc0 0x7fd438071470 unknown :-1 s=CLOSED pgs=3614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fd44c101f90 0x7fd44c10c960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fd44c1015f0 0x7fd44c112cb0 unknown :-1 s=CLOSED pgs=2757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.641 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 --2- 172.21.15.107:0/2558947535 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fd44c0fbd30 0x7fd44c112770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.642 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 >> 172.21.15.107:0/2558947535 conn(0x7fd44c072510 msgr2=0x7fd44c0fa7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:11.642 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 shutdown_connections 2024-09-10T19:11:11.642 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.638+0000 7fd45435d640 1 -- 172.21.15.107:0/2558947535 wait complete. 2024-09-10T19:11:11.649 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool application enable 6328337b-b283-4924-8680-a2da095e48b3 rados 2024-09-10T19:11:11.729 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.724+0000 7f06ff95d640 1 -- 172.21.15.107:0/2215074636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 msgr2=0x7f06f80fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.724+0000 7f06ff95d640 1 --2- 172.21.15.107:0/2215074636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f80fb650 secure :-1 s=READY pgs=2758 cs=0 l=1 rev1=1 crypto rx=0x7f06e8004950 tx=0x7f06e8009da0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- 172.21.15.107:0/2215074636 shutdown_connections 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- 172.21.15.107:0/2215074636 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 0x7f06f81083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- 172.21.15.107:0/2215074636 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f80fb650 unknown :-1 s=CLOSED pgs=2758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- 172.21.15.107:0/2215074636 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- 172.21.15.107:0/2215074636 >> 172.21.15.107:0/2215074636 conn(0x7f06f8072530 msgr2=0x7f06f80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- 172.21.15.107:0/2215074636 shutdown_connections 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- 172.21.15.107:0/2215074636 wait complete. 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 Processor -- start 2024-09-10T19:11:11.730 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- start start 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 0x7f06f810dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f810e720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f8118f20 con 0x7f06f81015f0 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06f8119090 con 0x7f06f80fbd30 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06ff95d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06f8119250 con 0x7f06f8101f90 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06fcef8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06fdefa640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f810e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.725+0000 7f06fcef8640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48730/0 (socket says 172.21.15.107:48730) 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fcef8640 1 -- 172.21.15.107:0/3718558406 learned_addr learned my addr 172.21.15.107:0/3718558406 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fd6f9640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 0x7f06f810dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 -- 172.21.15.107:0/3718558406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 msgr2=0x7f06f810dca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:11.731 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 0x7f06f810dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 -- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 msgr2=0x7f06f810e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06f4005040 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fcef8640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06fdefa640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f810e720 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f06f000bb00 tx=0x7f06f00096d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f000c9e0 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f06f0014410 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06e8005040 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06f00133c0 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.726+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06f8119870 con 0x7f06f8101f90 2024-09-10T19:11:11.732 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.727+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f06f8065720 con 0x7f06f8101f90 2024-09-10T19:11:11.733 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.728+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f06f000cb40 con 0x7f06f8101f90 2024-09-10T19:11:11.733 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.728+0000 7f06edffb640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 0x7f06e4071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:11.733 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.728+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(754..754 src has 1..754) v4 ==== 9443+0+0 (secure 0 0 0) 0x7f06f008f2a0 con 0x7f06f8101f90 2024-09-10T19:11:11.733 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.728+0000 7f06fd6f9640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 0x7f06e4071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:11.734 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.729+0000 7f06fd6f9640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 0x7f06e4071470 secure :-1 s=READY pgs=3615 cs=0 l=1 rev1=1 crypto rx=0x7f06f4007fd0 tx=0x7f06f4009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:11.735 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.732+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f06f004d2a0 con 0x7f06f8101f90 2024-09-10T19:11:11.891 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:11.888+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"} v 0) v1 -- 0x7f06f810a880 con 0x7f06f8101f90 2024-09-10T19:11:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:11 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:11.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:11 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:11 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:11.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:11 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:11 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:11.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:11 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.545 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:12.544+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]=0 enabled application 'rados' on pool '6328337b-b283-4924-8680-a2da095e48b3' v755) v1 ==== 213+0+0 (secure 0 0 0) 0x7f06f005b520 con 0x7f06f8101f90 2024-09-10T19:11:12.645 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:12.643+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"} v 0) v1 -- 0x7f06f804bf60 con 0x7f06f8101f90 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]': finished 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:12.826 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:11:12.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]': finished 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: pgmap v1210: 176 pgs: 176 active+clean; 732 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]': finished 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: osdmap e754: 8 total, 8 up, 8 in 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/2558947535' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pg_num": 12}]: dispatch 2024-09-10T19:11:12.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:12.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:12 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.174 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:13.556 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.554+0000 7f06edffb640 1 -- 172.21.15.107:0/3718558406 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]=0 enabled application 'rados' on pool '6328337b-b283-4924-8680-a2da095e48b3' v756) v1 ==== 213+0+0 (secure 0 0 0) 0x7f06f005aec0 con 0x7f06f8101f90 2024-09-10T19:11:13.556 INFO:tasks.workunit.client.0.smithi107.stderr:enabled application 'rados' on pool '6328337b-b283-4924-8680-a2da095e48b3' 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 msgr2=0x7f06e4071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 0x7f06e4071470 secure :-1 s=READY pgs=3615 cs=0 l=1 rev1=1 crypto rx=0x7f06f4007fd0 tx=0x7f06f4009da0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 msgr2=0x7f06f810e720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f810e720 secure :-1 s=READY pgs=2759 cs=0 l=1 rev1=1 crypto rx=0x7f06f000bb00 tx=0x7f06f00096d0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 shutdown_connections 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f06e406efc0 0x7f06e4071470 unknown :-1 s=CLOSED pgs=3615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f06f8101f90 0x7f06f810e720 unknown :-1 s=CLOSED pgs=2759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f06f81015f0 0x7f06f810e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 --2- 172.21.15.107:0/3718558406 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f06f80fbd30 0x7f06f810dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.560 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 >> 172.21.15.107:0/3718558406 conn(0x7f06f8072530 msgr2=0x7f06f80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:13.560 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 shutdown_connections 2024-09-10T19:11:13.560 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.557+0000 7f06ff95d640 1 -- 172.21.15.107:0/3718558406 wait complete. 2024-09-10T19:11:13.570 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota 6328337b-b283-4924-8680-a2da095e48b3 max_objects 10 2024-09-10T19:11:13.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 -- 172.21.15.107:0/2295328243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 msgr2=0x7f633c0fb650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:13.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 --2- 172.21.15.107:0/2295328243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fb650 secure :-1 s=READY pgs=2760 cs=0 l=1 rev1=1 crypto rx=0x7f6330004950 tx=0x7f6330009da0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.655 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 -- 172.21.15.107:0/2295328243 shutdown_connections 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 --2- 172.21.15.107:0/2295328243 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c1083d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 --2- 172.21.15.107:0/2295328243 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fb650 unknown :-1 s=CLOSED pgs=2760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 --2- 172.21.15.107:0/2295328243 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c1019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 -- 172.21.15.107:0/2295328243 >> 172.21.15.107:0/2295328243 conn(0x7f633c072510 msgr2=0x7f633c0f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 -- 172.21.15.107:0/2295328243 shutdown_connections 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.651+0000 7f634279d640 1 -- 172.21.15.107:0/2295328243 wait complete. 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 Processor -- start 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 -- start start 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:13.656 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c0fd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c11b030 con 0x7f633c0fbd30 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c11b1a0 con 0x7f633c1015f0 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f634279d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c11b360 con 0x7f633c101f90 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f6340d3a640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:48744/0 (socket says 172.21.15.107:48744) 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 -- 172.21.15.107:0/3542415737 learned_addr learned my addr 172.21.15.107:0/3542415737 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633b7fe640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c0fd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 -- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 msgr2=0x7f633c0fdcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:13.657 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fdcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 -- 172.21.15.107:0/3542415737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 msgr2=0x7f633c0fd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c0fd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633bfff640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63340098b0 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f6340d3a640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.652+0000 7f633b7fe640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c0fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f633bfff640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f632c002b00 tx=0x7f632c004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c007c30 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6330005040 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f632c00f040 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f632c009340 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.653+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c11b9d0 con 0x7f633c0fbd30 2024-09-10T19:11:13.658 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.654+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c04bf60 con 0x7f633c0fbd30 2024-09-10T19:11:13.659 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.654+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f632c009770 con 0x7f633c0fbd30 2024-09-10T19:11:13.659 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.655+0000 7f6338ff9640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 0x7f6324071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:11:13.659 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.655+0000 7f633b7fe640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 0x7f6324071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:11:13.660 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.655+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(756..756 src has 1..756) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f632c08f020 con 0x7f633c0fbd30 2024-09-10T19:11:13.660 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.655+0000 7f633b7fe640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 0x7f6324071470 secure :-1 s=READY pgs=3616 cs=0 l=1 rev1=1 crypto rx=0x7f633c0fe910 tx=0x7f6330009da0 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:11:13.661 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.659+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f632c05b170 con 0x7f633c0fbd30 2024-09-10T19:11:13.816 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:13.814+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f633c0fe6c0 con 0x7f633c0fbd30 2024-09-10T19:11:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:13 smithi162 ceph-mon[33228]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T19:11:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:13 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:13 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[29404]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[40355]: osdmap e755: 8 total, 8 up, 8 in 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3718558406' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:13.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:13 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]: dispatch 2024-09-10T19:11:14.559 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:14.557+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 6328337b-b283-4924-8680-a2da095e48b3 v757) v1 ==== 223+0+0 (secure 0 0 0) 0x7f632c05ab10 con 0x7f633c0fbd30 2024-09-10T19:11:14.653 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:14.651+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"} v 0) v1 -- 0x7f633c10a800 con 0x7f633c0fbd30 2024-09-10T19:11:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:14 smithi162 ceph-mon[33228]: pgmap v1213: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.3 KiB/s wr, 1 op/s 2024-09-10T19:11:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:14 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:14 smithi162 ceph-mon[33228]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T19:11:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:14 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:14.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:14.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[29404]: pgmap v1213: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.3 KiB/s wr, 1 op/s 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[29404]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[40355]: pgmap v1213: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.3 KiB/s wr, 1 op/s 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool application enable", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "app": "rados"}]': finished 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[40355]: osdmap e756: 8 total, 8 up, 8 in 2024-09-10T19:11:14.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:14 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:15.565 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.564+0000 7f6338ff9640 1 -- 172.21.15.107:0/3542415737 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]=0 set-quota max_objects = 10 for pool 6328337b-b283-4924-8680-a2da095e48b3 v758) v1 ==== 223+0+0 (secure 0 0 0) 0x7f632c05acf0 con 0x7f633c0fbd30 2024-09-10T19:11:15.631 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 10 for pool 6328337b-b283-4924-8680-a2da095e48b3 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 msgr2=0x7f6324071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 0x7f6324071470 secure :-1 s=READY pgs=3616 cs=0 l=1 rev1=1 crypto rx=0x7f633c0fe910 tx=0x7f6330009da0 comp rx=0 tx=0).stop 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 msgr2=0x7f633c0fd230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 secure :-1 s=READY pgs=2425 cs=0 l=1 rev1=1 crypto rx=0x7f632c002b00 tx=0x7f632c004600 comp rx=0 tx=0).stop 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 shutdown_connections 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f632406efc0 0x7f6324071470 unknown :-1 s=CLOSED pgs=3616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f633c101f90 0x7f633c0fdcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f633c1015f0 0x7f633c0fd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:15.633 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 --2- 172.21.15.107:0/3542415737 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f633c0fbd30 0x7f633c0fd230 unknown :-1 s=CLOSED pgs=2425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:11:15.634 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 >> 172.21.15.107:0/3542415737 conn(0x7f633c072510 msgr2=0x7f633c0734e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:11:15.634 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 shutdown_connections 2024-09-10T19:11:15.634 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:11:15.566+0000 7f634279d640 1 -- 172.21.15.107:0/3542415737 wait complete. 2024-09-10T19:11:15.634 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-10T19:11:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:15 smithi162 ceph-mon[33228]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T19:11:15.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:15 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[29404]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T19:11:15.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[40355]: osdmap e757: 8 total, 8 up, 8 in 2024-09-10T19:11:15.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:15 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]: dispatch 2024-09-10T19:11:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:16 smithi162 ceph-mon[33228]: pgmap v1216: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T19:11:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:16 smithi162 ceph-mon[33228]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T19:11:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:16.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[29404]: pgmap v1216: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[29404]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[40355]: pgmap v1216: 188 pgs: 12 creating+peering, 176 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 6.5 KiB/s wr, 1 op/s 2024-09-10T19:11:16.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3542415737' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "field": "max_objects", "val": "10"}]': finished 2024-09-10T19:11:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[40355]: osdmap e758: 8 total, 8 up, 8 in 2024-09-10T19:11:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:16.967 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:18 smithi162 ceph-mon[33228]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:11:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:18 smithi107 ceph-mon[29404]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:11:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:18 smithi107 ceph-mon[40355]: pgmap v1218: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 1 op/s 2024-09-10T19:11:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:19] "GET /metrics HTTP/1.1" 200 228619 "" "Prometheus/2.43.0" 2024-09-10T19:11:20.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:20 smithi162 ceph-mon[33228]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:20 smithi107 ceph-mon[29404]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:20 smithi107 ceph-mon[40355]: pgmap v1219: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:22 smithi162 ceph-mon[33228]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 0 op/s 2024-09-10T19:11:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:22 smithi107 ceph-mon[29404]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 0 op/s 2024-09-10T19:11:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:22 smithi107 ceph-mon[40355]: pgmap v1220: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 655 B/s rd, 0 op/s 2024-09-10T19:11:23.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:22 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:24 smithi162 ceph-mon[33228]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:11:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:24 smithi107 ceph-mon[29404]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:11:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:24 smithi107 ceph-mon[40355]: pgmap v1221: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:11:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:26 smithi162 ceph-mon[33228]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:26 smithi107 ceph-mon[29404]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:26 smithi107 ceph-mon[40355]: pgmap v1222: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:28 smithi162 ceph-mon[33228]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:11:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:28 smithi107 ceph-mon[29404]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:11:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:28 smithi107 ceph-mon[40355]: pgmap v1223: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.3 KiB/s rd, 1 op/s 2024-09-10T19:11:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:29] "GET /metrics HTTP/1.1" 200 228619 "" "Prometheus/2.43.0" 2024-09-10T19:11:30.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:30 smithi162 ceph-mon[33228]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:30.925 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:30.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:30 smithi107 ceph-mon[29404]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:30 smithi107 ceph-mon[40355]: pgmap v1224: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:32 smithi162 ceph-mon[33228]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:32 smithi107 ceph-mon[29404]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:32 smithi107 ceph-mon[40355]: pgmap v1225: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:33.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:34 smithi162 ceph-mon[33228]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:34 smithi107 ceph-mon[29404]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:34 smithi107 ceph-mon[40355]: pgmap v1226: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:36 smithi162 ceph-mon[33228]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:36.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:36 smithi107 ceph-mon[29404]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:36.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:36 smithi107 ceph-mon[40355]: pgmap v1227: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:38 smithi162 ceph-mon[33228]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:38.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:38 smithi107 ceph-mon[29404]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:38.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:38 smithi107 ceph-mon[40355]: pgmap v1228: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:39] "GET /metrics HTTP/1.1" 200 228611 "" "Prometheus/2.43.0" 2024-09-10T19:11:40.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:40 smithi162 ceph-mon[33228]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:40.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:40 smithi107 ceph-mon[29404]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:40.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:40 smithi107 ceph-mon[40355]: pgmap v1229: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:42 smithi162 ceph-mon[33228]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:42 smithi107 ceph-mon[29404]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:42 smithi107 ceph-mon[40355]: pgmap v1230: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:43.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:44 smithi162 ceph-mon[33228]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:44 smithi107 ceph-mon[29404]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:44.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:44 smithi107 ceph-mon[40355]: pgmap v1231: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:11:45.580 INFO:tasks.workunit.client.0.smithi107.stderr:++ seq 1 10 2024-09-10T19:11:45.581 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.581 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj1 /etc/passwd 2024-09-10T19:11:45.618 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.618 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj2 /etc/passwd 2024-09-10T19:11:45.651 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.651 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj3 /etc/passwd 2024-09-10T19:11:45.683 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.683 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj4 /etc/passwd 2024-09-10T19:11:45.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.715 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj5 /etc/passwd 2024-09-10T19:11:45.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.745 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj6 /etc/passwd 2024-09-10T19:11:45.775 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.776 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj7 /etc/passwd 2024-09-10T19:11:45.806 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.806 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj8 /etc/passwd 2024-09-10T19:11:45.836 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.837 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj9 /etc/passwd 2024-09-10T19:11:45.867 INFO:tasks.workunit.client.0.smithi107.stderr:+ for f in `seq 1 10` 2024-09-10T19:11:45.867 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p 6328337b-b283-4924-8680-a2da095e48b3 put obj10 /etc/passwd 2024-09-10T19:11:45.898 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-10T19:11:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:46 smithi162 ceph-mon[33228]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:46 smithi107 ceph-mon[29404]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:46 smithi107 ceph-mon[40355]: pgmap v1232: 188 pgs: 188 active+clean; 757 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:11:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:11:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:48 smithi162 ceph-mon[33228]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T19:11:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:48 smithi107 ceph-mon[29404]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T19:11:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:48 smithi107 ceph-mon[40355]: pgmap v1233: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 11 KiB/s wr, 2 op/s 2024-09-10T19:11:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:49] "GET /metrics HTTP/1.1" 200 228571 "" "Prometheus/2.43.0" 2024-09-10T19:11:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:49 smithi162 ceph-mon[33228]: pool '6328337b-b283-4924-8680-a2da095e48b3' is full (reached quota's max_objects: 10) 2024-09-10T19:11:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:49 smithi162 ceph-mon[33228]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:11:49.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:49 smithi162 ceph-mon[33228]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T19:11:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[40355]: pool '6328337b-b283-4924-8680-a2da095e48b3' is full (reached quota's max_objects: 10) 2024-09-10T19:11:49.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[40355]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:11:49.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[40355]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T19:11:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[29404]: pool '6328337b-b283-4924-8680-a2da095e48b3' is full (reached quota's max_objects: 10) 2024-09-10T19:11:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[29404]: Health check failed: 1 pool(s) full (POOL_FULL) 2024-09-10T19:11:49.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:49 smithi107 ceph-mon[29404]: osdmap e759: 8 total, 8 up, 8 in 2024-09-10T19:11:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:50 smithi162 ceph-mon[33228]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:50.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:50 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:50.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:11:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:11:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:50 smithi107 ceph-mon[40355]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:50 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:50 smithi107 ceph-mon[29404]: pgmap v1235: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:50 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:11:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:11:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:11:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:51.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:11:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:11:51.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:11:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:11:51.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:11:52.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:52 smithi162 ceph-mon[33228]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:52.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:52 smithi107 ceph-mon[40355]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:52 smithi107 ceph-mon[29404]: pgmap v1236: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:53.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:11:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:11:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:54 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:54.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:54 smithi162 ceph-mon[33228]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:54 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:54.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:54 smithi107 ceph-mon[29404]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:54 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:11:54.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:54 smithi107 ceph-mon[40355]: pgmap v1237: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:56.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:56 smithi162 ceph-mon[33228]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:56.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:56 smithi107 ceph-mon[29404]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:56.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:56 smithi107 ceph-mon[40355]: pgmap v1238: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 13 KiB/s wr, 1 op/s 2024-09-10T19:11:58.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:11:58 smithi162 ceph-mon[33228]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:58.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:11:58 smithi107 ceph-mon[29404]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:58.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:11:58 smithi107 ceph-mon[40355]: pgmap v1239: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:11:59.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:11:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:11:59] "GET /metrics HTTP/1.1" 200 228571 "" "Prometheus/2.43.0" 2024-09-10T19:12:00.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:00 smithi162 ceph-mon[33228]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s 2024-09-10T19:12:00.924 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:00 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:00] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:00.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:00 smithi107 ceph-mon[29404]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s 2024-09-10T19:12:00.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:00 smithi107 ceph-mon[40355]: pgmap v1240: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1009 B/s rd, 0 op/s 2024-09-10T19:12:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:02 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:02.424 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:02 smithi162 ceph-mon[33228]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:02.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[29404]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:02.466 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:02 smithi107 ceph-mon[40355]: pgmap v1241: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:03.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:03 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:04 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:04.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:04 smithi107 ceph-mon[40355]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:04 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:04.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:04 smithi107 ceph-mon[29404]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:04 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:04.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:04 smithi162 ceph-mon[33228]: pgmap v1242: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:06.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:06 smithi107 ceph-mon[29404]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:06.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:06 smithi107 ceph-mon[40355]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:06.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:06 smithi162 ceph-mon[33228]: pgmap v1243: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:08.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:08 smithi107 ceph-mon[40355]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:08.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:08 smithi107 ceph-mon[29404]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:08.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:08 smithi162 ceph-mon[33228]: pgmap v1244: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:09.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:09 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:09] "GET /metrics HTTP/1.1" 200 228674 "" "Prometheus/2.43.0" 2024-09-10T19:12:10.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:10 smithi107 ceph-mon[29404]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:10.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:10 smithi107 ceph-mon[40355]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:10.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:10 smithi162 ceph-mon[33228]: pgmap v1245: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:11.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:10 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:10] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:12.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:12 smithi107 ceph-mon[29404]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:12.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:12 smithi107 ceph-mon[40355]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:12.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:12 smithi162 ceph-mon[33228]: pgmap v1246: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:13.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:13 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:14.715 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:14 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:14.716 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:14 smithi107 ceph-mon[29404]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:14 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:14.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:14 smithi107 ceph-mon[40355]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:14 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:14.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:14 smithi162 ceph-mon[33228]: pgmap v1247: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:15.899 INFO:tasks.workunit.client.0.smithi107.stderr:+ rados -p b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f put threemore /etc/passwd 2024-09-10T19:12:15.937 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_bytes 0 2024-09-10T19:12:16.024 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.017+0000 7f38ec4c0640 1 -- 172.21.15.107:0/1588303156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 msgr2=0x7f38e41095b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.017+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/1588303156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e41095b0 secure :-1 s=READY pgs=2777 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009d50 tx=0x7f38dc004c50 comp rx=0 tx=0).stop 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- 172.21.15.107:0/1588303156 shutdown_connections 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/1588303156 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e41095b0 unknown :-1 s=CLOSED pgs=2777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/1588303156 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e40fac70 0x7f38e40fb0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/1588303156 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e41016d0 0x7f38e4101aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- 172.21.15.107:0/1588303156 >> 172.21.15.107:0/1588303156 conn(0x7f38e4074b80 msgr2=0x7f38e40f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:16.025 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- 172.21.15.107:0/1588303156 shutdown_connections 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- 172.21.15.107:0/1588303156 wait complete. 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 Processor -- start 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- start start 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e4112d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e41016d0 0x7f38e410c980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410cfd0 con 0x7f38e40fac70 2024-09-10T19:12:16.026 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410d140 con 0x7f38e41016d0 2024-09-10T19:12:16.027 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ec4c0640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38e410d300 con 0x7f38e40fb620 2024-09-10T19:12:16.027 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38e9a5b640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e4112d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:16.027 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ea25c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:16.027 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ea25c640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36870/0 (socket says 172.21.15.107:36870) 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.018+0000 7f38ea25c640 1 -- 172.21.15.107:0/3004622096 learned_addr learned my addr 172.21.15.107:0/3004622096 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38eaa5d640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e41016d0 0x7f38e410c980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 -- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 msgr2=0x7f38e4112d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e4112d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 -- 172.21.15.107:0/3004622096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e41016d0 msgr2=0x7f38e410c980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e41016d0 0x7f38e410c980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38d4005040 con 0x7f38e40fac70 2024-09-10T19:12:16.028 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38e9a5b640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e4112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ea25c640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009550 tx=0x7f38d4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d400e930 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f38d400f040 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38dc0098b0 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38d4013b10 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.019+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38e4119670 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.020+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38e40fdf90 con 0x7f38e40fac70 2024-09-10T19:12:16.029 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.021+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f38d400ea90 con 0x7f38e40fac70 2024-09-10T19:12:16.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.021+0000 7f38e2ffd640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 0x7f38cc071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:16.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.021+0000 7f38e9a5b640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 0x7f38cc071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:16.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.021+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(759..759 src has 1..759) v4 ==== 9456+0+0 (secure 0 0 0) 0x7f38d408ef80 con 0x7f38e40fac70 2024-09-10T19:12:16.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.022+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=760}) v3 -- 0x7f38cc07afe0 con 0x7f38e40fac70 2024-09-10T19:12:16.030 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.022+0000 7f38e9a5b640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 0x7f38cc071470 secure :-1 s=READY pgs=3628 cs=0 l=1 rev1=1 crypto rx=0x7f38d8004fd0 tx=0x7f38d800b040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:16.031 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.026+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f38d405b180 con 0x7f38e40fac70 2024-09-10T19:12:16.204 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.203+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f38e410de40 con 0x7f38e40fac70 2024-09-10T19:12:16.452 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.450+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 7 ==== osd_map(760..760 src has 1..760) v4 ==== 620+0+0 (secure 0 0 0) 0x7f38d4013400 con 0x7f38e40fac70 2024-09-10T19:12:16.452 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.450+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=761}) v3 -- 0x7f38cc07bba0 con 0x7f38e40fac70 2024-09-10T19:12:16.455 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.453+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v760) v1 ==== 217+0+0 (secure 0 0 0) 0x7f38d405ab20 con 0x7f38e40fac70 2024-09-10T19:12:16.551 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:16.549+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"} v 0) v1 -- 0x7f38e410ba50 con 0x7f38e40fac70 2024-09-10T19:12:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[40355]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:16.716 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[29404]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:16.717 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:16 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:16 smithi162 ceph-mon[33228]: pgmap v1248: 188 pgs: 188 active+clean; 882 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:16 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:16.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:16 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:17.466 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.464+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 9 ==== osd_map(761..761 src has 1..761) v4 ==== 620+0+0 (secure 0 0 0) 0x7f38d405aec0 con 0x7f38e40fac70 2024-09-10T19:12:17.466 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.465+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=762}) v3 -- 0x7f38cc07be70 con 0x7f38e40fac70 2024-09-10T19:12:17.470 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.468+0000 7f38e2ffd640 1 -- 172.21.15.107:0/3004622096 <== mon.0 v2:172.21.15.107:3300/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]=0 set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v761) v1 ==== 217+0+0 (secure 0 0 0) 0x7f38d405ad00 con 0x7f38e40fac70 2024-09-10T19:12:17.470 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_bytes = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 msgr2=0x7f38cc071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 0x7f38cc071470 secure :-1 s=READY pgs=3628 cs=0 l=1 rev1=1 crypto rx=0x7f38d8004fd0 tx=0x7f38d800b040 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 msgr2=0x7f38e41127d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7f38d4009550 tx=0x7f38d4009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 shutdown_connections 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f38cc06efc0 0x7f38cc071470 unknown :-1 s=CLOSED pgs=3628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f38e41016d0 0x7f38e410c980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f38e40fb620 0x7f38e4112d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 --2- 172.21.15.107:0/3004622096 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f38e40fac70 0x7f38e41127d0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 >> 172.21.15.107:0/3004622096 conn(0x7f38e4074b80 msgr2=0x7f38e4070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 shutdown_connections 2024-09-10T19:12:17.474 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.472+0000 7f38ec4c0640 1 -- 172.21.15.107:0/3004622096 wait complete. 2024-09-10T19:12:17.483 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool set-quota b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f max_objects 0 2024-09-10T19:12:17.567 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 -- 172.21.15.107:0/3982373130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fd6f0 msgr2=0x7fba300fdac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:17.567 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 --2- 172.21.15.107:0/3982373130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fd6f0 0x7fba300fdac0 secure :-1 s=READY pgs=2778 cs=0 l=1 rev1=1 crypto rx=0x7fba20004950 tx=0x7fba20009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.567 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 -- 172.21.15.107:0/3982373130 shutdown_connections 2024-09-10T19:12:17.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 --2- 172.21.15.107:0/3982373130 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 0x7fba30109080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 --2- 172.21.15.107:0/3982373130 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fe090 0x7fba30106760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 --2- 172.21.15.107:0/3982373130 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fd6f0 0x7fba300fdac0 unknown :-1 s=CLOSED pgs=2778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 -- 172.21.15.107:0/3982373130 >> 172.21.15.107:0/3982373130 conn(0x7fba30072510 msgr2=0x7fba300f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:17.568 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 -- 172.21.15.107:0/3982373130 shutdown_connections 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.559+0000 7fba35434640 1 -- 172.21.15.107:0/3982373130 wait complete. 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 Processor -- start 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 -- start start 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:17.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 0x7fba3010cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:17.570 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3010d2d0 con 0x7fba300fd6f0 2024-09-10T19:12:17.570 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba3010d440 con 0x7fba30106ca0 2024-09-10T19:12:17.570 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba35434640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba3010d600 con 0x7fba300fe090 2024-09-10T19:12:17.570 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:17.570 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:17.571 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2effd640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:36884/0 (socket says 172.21.15.107:36884) 2024-09-10T19:12:17.571 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3301/0 says I am v2:172.21.15.107:40784/0 (socket says 172.21.15.107:40784) 2024-09-10T19:12:17.571 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2effd640 1 -- 172.21.15.107:0/3565710017 learned_addr learned my addr 172.21.15.107:0/3565710017 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:12:17.571 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 -- 172.21.15.107:0/3565710017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 msgr2=0x7fba3010cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:17.571 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2f7fe640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 0x7fba3010cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:17.572 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 0x7fba3010cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.572 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 -- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 msgr2=0x7fba30112af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:17.572 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:17.572 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2e7fc640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba24005040 con 0x7fba300fe090 2024-09-10T19:12:17.572 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.560+0000 7fba2effd640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba2e7fc640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fba24009380 tx=0x7fba24009da0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24007b00 con 0x7fba300fe090 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fba2400f040 con 0x7fba300fe090 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba20005040 con 0x7fba300fe090 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba24013a50 con 0x7fba300fe090 2024-09-10T19:12:17.573 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.561+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba30119940 con 0x7fba300fe090 2024-09-10T19:12:17.574 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.562+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba30066510 con 0x7fba300fe090 2024-09-10T19:12:17.574 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.562+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fba2400e8d0 con 0x7fba300fe090 2024-09-10T19:12:17.574 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.562+0000 7fba177fe640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 0x7fba18071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:17.574 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.563+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 5 ==== osd_map(761..761 src has 1..761) v4 ==== 9456+0+0 (secure 0 0 0) 0x7fba2408eed0 con 0x7fba300fe090 2024-09-10T19:12:17.574 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.563+0000 7fba2effd640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 0x7fba18071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:17.575 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.563+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=762}) v3 -- 0x7fba1807b060 con 0x7fba300fe090 2024-09-10T19:12:17.575 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.563+0000 7fba2effd640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 0x7fba18071470 secure :-1 s=READY pgs=3629 cs=0 l=1 rev1=1 crypto rx=0x7fba20004540 tx=0x7fba20035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:17.576 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.566+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fba2405b160 con 0x7fba300fe090 2024-09-10T19:12:17.724 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:17.722+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fba3010b960 con 0x7fba300fe090 2024-09-10T19:12:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:17 smithi162 ceph-mon[33228]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T19:12:17.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:17 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[29404]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T19:12:17.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[40355]: osdmap e760: 8 total, 8 up, 8 in 2024-09-10T19:12:17.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:17 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]: dispatch 2024-09-10T19:12:18.468 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:18.466+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 7 ==== osd_map(762..762 src has 1..762) v4 ==== 620+0+0 (secure 0 0 0) 0x7fba2405aea0 con 0x7fba300fe090 2024-09-10T19:12:18.468 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:18.466+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=763}) v3 -- 0x7fba1807bc90 con 0x7fba300fe090 2024-09-10T19:12:18.471 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:18.469+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v762) v1 ==== 221+0+0 (secure 0 0 0) 0x7fba2405ab00 con 0x7fba300fe090 2024-09-10T19:12:18.569 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:18.568+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_command({"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"} v 0) v1 -- 0x7fba300654a0 con 0x7fba300fe090 2024-09-10T19:12:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:18 smithi162 ceph-mon[33228]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T19:12:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:18 smithi162 ceph-mon[33228]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T19:12:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:18 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:18.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:18 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[29404]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T19:12:18.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[29404]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[40355]: pgmap v1250: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 1.3 KiB/s wr, 1 op/s 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3004622096' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_bytes", "val": "0"}]': finished 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[40355]: osdmap e761: 8 total, 8 up, 8 in 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:18.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:18 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:19] "GET /metrics HTTP/1.1" 200 228651 "" "Prometheus/2.43.0" 2024-09-10T19:12:19.484 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.483+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 9 ==== osd_map(763..763 src has 1..763) v4 ==== 620+0+0 (secure 0 0 0) 0x7fba2405ba90 con 0x7fba300fe090 2024-09-10T19:12:19.485 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.483+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_subscribe({osdmap=764}) v3 -- 0x7fba1807c160 con 0x7fba300fe090 2024-09-10T19:12:19.488 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.486+0000 7fba177fe640 1 -- 172.21.15.107:0/3565710017 <== mon.2 v2:172.21.15.107:3301/0 10 ==== mon_command_ack([{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]=0 set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f v763) v1 ==== 221+0+0 (secure 0 0 0) 0x7fba2405ace0 con 0x7fba300fe090 2024-09-10T19:12:19.488 INFO:tasks.workunit.client.0.smithi107.stderr:set-quota max_objects = 0 for pool b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 msgr2=0x7fba18071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 0x7fba18071470 secure :-1 s=READY pgs=3629 cs=0 l=1 rev1=1 crypto rx=0x7fba20004540 tx=0x7fba20035040 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 msgr2=0x7fba30113030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 secure :-1 s=READY pgs=2779 cs=0 l=1 rev1=1 crypto rx=0x7fba24009380 tx=0x7fba24009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 shutdown_connections 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fba1806efc0 0x7fba18071470 unknown :-1 s=CLOSED pgs=3629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fba30106ca0 0x7fba3010cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fba300fe090 0x7fba30113030 unknown :-1 s=CLOSED pgs=2779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 --2- 172.21.15.107:0/3565710017 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fba300fd6f0 0x7fba30112af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 >> 172.21.15.107:0/3565710017 conn(0x7fba30072510 msgr2=0x7fba30073270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 shutdown_connections 2024-09-10T19:12:19.492 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:19.490+0000 7fba35434640 1 -- 172.21.15.107:0/3565710017 wait complete. 2024-09-10T19:12:19.502 INFO:tasks.workunit.client.0.smithi107.stderr:+ sleep 30 2024-09-10T19:12:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:19 smithi162 ceph-mon[33228]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T19:12:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:19 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:19 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[29404]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[40355]: osdmap e762: 8 total, 8 up, 8 in 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/3565710017' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:19.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:19 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd=[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]: dispatch 2024-09-10T19:12:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:20 smithi162 ceph-mon[33228]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T19:12:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:20 smithi162 ceph-mon[33228]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:20.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:20 smithi162 ceph-mon[33228]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T19:12:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[29404]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T19:12:20.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[29404]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:20.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[29404]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T19:12:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[40355]: pgmap v1253: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 2.2 KiB/s wr, 0 op/s 2024-09-10T19:12:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[40355]: from='client.? ' entity='client.admin' cmd='[{"prefix": "osd pool set-quota", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "field": "max_objects", "val": "0"}]': finished 2024-09-10T19:12:20.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:20 smithi107 ceph-mon[40355]: osdmap e763: 8 total, 8 up, 8 in 2024-09-10T19:12:21.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:20 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:20] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:22.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:22 smithi162 ceph-mon[33228]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T19:12:22.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:22 smithi107 ceph-mon[29404]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T19:12:22.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:22 smithi107 ceph-mon[40355]: pgmap v1255: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.0 KiB/s rd, 2.6 KiB/s wr, 1 op/s 2024-09-10T19:12:23.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:23 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:24 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:24.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:24 smithi162 ceph-mon[33228]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:24.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:24 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:24.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:24 smithi107 ceph-mon[29404]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:24 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:24.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:24 smithi107 ceph-mon[40355]: pgmap v1256: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:26.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:26 smithi162 ceph-mon[33228]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 646 B/s rd, 0 op/s 2024-09-10T19:12:26.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:26 smithi107 ceph-mon[29404]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 646 B/s rd, 0 op/s 2024-09-10T19:12:26.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:26 smithi107 ceph-mon[40355]: pgmap v1257: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 646 B/s rd, 0 op/s 2024-09-10T19:12:28.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:28 smithi162 ceph-mon[33228]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:12:28.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:28 smithi107 ceph-mon[29404]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:12:28.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:28 smithi107 ceph-mon[40355]: pgmap v1258: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.1 KiB/s rd, 1 op/s 2024-09-10T19:12:29.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:29 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:29] "GET /metrics HTTP/1.1" 200 228651 "" "Prometheus/2.43.0" 2024-09-10T19:12:30.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:30 smithi162 ceph-mon[33228]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:30.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:30 smithi107 ceph-mon[29404]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:30.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:30 smithi107 ceph-mon[40355]: pgmap v1259: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:31.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:30] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:31.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:31 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:31.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:31 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:31.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:31 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:32.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:32 smithi162 ceph-mon[33228]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-10T19:12:32.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:32 smithi107 ceph-mon[29404]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-10T19:12:32.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:32 smithi107 ceph-mon[40355]: pgmap v1260: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 860 B/s rd, 0 op/s 2024-09-10T19:12:33.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:34 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:34.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:34 smithi162 ceph-mon[33228]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:34.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:34 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:34.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:34 smithi107 ceph-mon[29404]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:34 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:34.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:34 smithi107 ceph-mon[40355]: pgmap v1261: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:36.768 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:36 smithi107 ceph-mon[29404]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:36.768 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:36 smithi107 ceph-mon[40355]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:36.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:36 smithi162 ceph-mon[33228]: pgmap v1262: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:38.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:38 smithi162 ceph-mon[33228]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:38.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:38 smithi107 ceph-mon[29404]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:38.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:38 smithi107 ceph-mon[40355]: pgmap v1263: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:39.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:39 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:39] "GET /metrics HTTP/1.1" 200 228665 "" "Prometheus/2.43.0" 2024-09-10T19:12:40.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:40 smithi162 ceph-mon[33228]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:40.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:40 smithi107 ceph-mon[29404]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:40.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:40 smithi107 ceph-mon[40355]: pgmap v1264: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:41.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:40] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:42.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:42 smithi162 ceph-mon[33228]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:42.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:42 smithi107 ceph-mon[29404]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:42.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:42 smithi107 ceph-mon[40355]: pgmap v1265: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:43.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:44 smithi162 ceph-mon[33228]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:44.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:44 smithi162 ceph-mon[33228]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:44 smithi107 ceph-mon[29404]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:44.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:44 smithi107 ceph-mon[29404]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:44 smithi107 ceph-mon[40355]: from='client.24376 -' entity='client.iscsi.iscsi.a' cmd=[{"prefix": "service status", "format": "json"}]: dispatch 2024-09-10T19:12:44.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:44 smithi107 ceph-mon[40355]: pgmap v1266: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:46 smithi162 ceph-mon[33228]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:12:46.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:46 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:46.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:46 smithi107 ceph-mon[29404]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:12:46.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:46 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:46 smithi107 ceph-mon[40355]: pgmap v1267: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 852 B/s rd, 0 op/s 2024-09-10T19:12:46.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:46 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "osd blocklist ls", "format": "json"}]: dispatch 2024-09-10T19:12:48.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:48 smithi162 ceph-mon[33228]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:48.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:48 smithi107 ceph-mon[29404]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:48.966 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:48 smithi107 ceph-mon[40355]: pgmap v1268: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-10T19:12:49.465 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:49 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:49] "GET /metrics HTTP/1.1" 200 228647 "" "Prometheus/2.43.0" 2024-09-10T19:12:49.503 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool delete b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f --yes-i-really-really-mean-it 2024-09-10T19:12:49.594 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 -- 172.21.15.107:0/2245135072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e80fbd30 msgr2=0x7fa8e81083d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/2245135072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e80fbd30 0x7fa8e81083d0 secure :-1 s=READY pgs=2780 cs=0 l=1 rev1=1 crypto rx=0x7fa8e0009cd0 tx=0x7fa8e0004d10 comp rx=0 tx=0).stop 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 -- 172.21.15.107:0/2245135072 shutdown_connections 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/2245135072 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e80fbd30 0x7fa8e81083d0 unknown :-1 s=CLOSED pgs=2780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/2245135072 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 0x7fa8e80fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/2245135072 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e81015f0 0x7fa8e81019c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 -- 172.21.15.107:0/2245135072 >> 172.21.15.107:0/2245135072 conn(0x7fa8e8072530 msgr2=0x7fa8e80f9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 -- 172.21.15.107:0/2245135072 shutdown_connections 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 -- 172.21.15.107:0/2245135072 wait complete. 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.591+0000 7fa8ee95d640 1 Processor -- start 2024-09-10T19:12:49.595 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 -- start start 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 0x7fa8e8111bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 0x7fa8e810b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e810bf60 con 0x7fa8e80fbd30 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e810c0d0 con 0x7fa8e8101f90 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ee95d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8e810c290 con 0x7fa8e81015f0 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e77fe640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 0x7fa8e8111bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53590/0 (socket says 172.21.15.107:53590) 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 -- 172.21.15.107:0/296933807 learned_addr learned my addr 172.21.15.107:0/296933807 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 -- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 msgr2=0x7fa8e8111bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:49.596 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 0x7fa8e8111bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 -- 172.21.15.107:0/296933807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 msgr2=0x7fa8e810b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8ecefa640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 0x7fa8e810b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 0x7fa8e810b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e7fff640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8d8005040 con 0x7fa8e80fbd30 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.592+0000 7fa8e77fe640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 0x7fa8e8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8e7fff640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009d70 tx=0x7fa8d8009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d8007bb0 con 0x7fa8e80fbd30 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7fa8d8012430 con 0x7fa8e80fbd30 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8e00098b0 con 0x7fa8e80fbd30 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8d801a400 con 0x7fa8e80fbd30 2024-09-10T19:12:49.597 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.593+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8e8064a90 con 0x7fa8e80fbd30 2024-09-10T19:12:49.598 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.594+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8e80fcd70 con 0x7fa8e80fbd30 2024-09-10T19:12:49.598 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.594+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7fa8d8010b10 con 0x7fa8e80fbd30 2024-09-10T19:12:49.598 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.595+0000 7fa8e4ff9640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 0x7fa8d0071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:49.598 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.595+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(763..763 src has 1..763) v4 ==== 9456+0+0 (secure 0 0 0) 0x7fa8d8091d60 con 0x7fa8e80fbd30 2024-09-10T19:12:49.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.595+0000 7fa8e77fe640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 0x7fa8d0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:49.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.595+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=764}) v3 -- 0x7fa8d007afe0 con 0x7fa8e80fbd30 2024-09-10T19:12:49.599 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.595+0000 7fa8e77fe640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 0x7fa8d0071470 secure :-1 s=READY pgs=3630 cs=0 l=1 rev1=1 crypto rx=0x7fa8e810cf50 tx=0x7fa8dc009040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:49.600 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.599+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7fa8d805df70 con 0x7fa8e80fbd30 2024-09-10T19:12:49.757 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:49.755+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8e810cd00 con 0x7fa8e80fbd30 2024-09-10T19:12:50.542 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.541+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 7 ==== osd_map(764..764 src has 1..764) v4 ==== 288+0+0 (secure 0 0 0) 0x7fa8d80127a0 con 0x7fa8e80fbd30 2024-09-10T19:12:50.542 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.541+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=765}) v3 -- 0x7fa8d007b930 con 0x7fa8e80fbd30 2024-09-10T19:12:50.549 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.547+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]=0 pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' removed v764) v1 ==== 248+0+0 (secure 0 0 0) 0x7fa8d805d910 con 0x7fa8e80fbd30 2024-09-10T19:12:50.649 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.647+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fa8e804bf60 con 0x7fa8e80fbd30 2024-09-10T19:12:50.649 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.648+0000 7fa8e4ff9640 1 -- 172.21.15.107:0/296933807 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]=0 pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' does not exist v764) v1 ==== 255+0+0 (secure 0 0 0) 0x7fa8d805daf0 con 0x7fa8e80fbd30 2024-09-10T19:12:50.649 INFO:tasks.workunit.client.0.smithi107.stderr:pool 'b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f' does not exist 2024-09-10T19:12:50.651 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 msgr2=0x7fa8d0071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:50.651 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 0x7fa8d0071470 secure :-1 s=READY pgs=3630 cs=0 l=1 rev1=1 crypto rx=0x7fa8e810cf50 tx=0x7fa8dc009040 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 msgr2=0x7fa8e8111690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7fa8d8009d70 tx=0x7fa8d8009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 shutdown_connections 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7fa8d006efc0 0x7fa8d0071470 unknown :-1 s=CLOSED pgs=3630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7fa8e8101f90 0x7fa8e810b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7fa8e81015f0 0x7fa8e8111bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 --2- 172.21.15.107:0/296933807 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7fa8e80fbd30 0x7fa8e8111690 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 >> 172.21.15.107:0/296933807 conn(0x7fa8e8072530 msgr2=0x7fa8e80ff170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 shutdown_connections 2024-09-10T19:12:50.652 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.650+0000 7fa8ee95d640 1 -- 172.21.15.107:0/296933807 wait complete. 2024-09-10T19:12:50.660 INFO:tasks.workunit.client.0.smithi107.stderr:+ ceph osd pool delete 6328337b-b283-4924-8680-a2da095e48b3 6328337b-b283-4924-8680-a2da095e48b3 --yes-i-really-really-mean-it 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- 172.21.15.107:0/3718959306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 msgr2=0x7f92f0101a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 --2- 172.21.15.107:0/3718959306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0101a00 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f92e0004950 tx=0x7f92e0009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- 172.21.15.107:0/3718959306 shutdown_connections 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 --2- 172.21.15.107:0/3718959306 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 0x7f92f0108430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 --2- 172.21.15.107:0/3718959306 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f00fbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 --2- 172.21.15.107:0/3718959306 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0101a00 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- 172.21.15.107:0/3718959306 >> 172.21.15.107:0/3718959306 conn(0x7f92f0074b80 msgr2=0x7f92f00f8fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- 172.21.15.107:0/3718959306 shutdown_connections 2024-09-10T19:12:50.740 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- 172.21.15.107:0/3718959306 wait complete. 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 Processor -- start 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.736+0000 7f92f499d640 1 -- start start 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 --2- >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 0x7f92f0112a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f010cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 -- --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010d1e0 con 0x7f92f0101630 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 -- --> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010d350 con 0x7f92f00fc470 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 -- --> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92f010d510 con 0x7f92f0101fd0 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92eed9d640 1 --2- >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f010cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 --2- >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.107:3300/0 says I am v2:172.21.15.107:53616/0 (socket says 172.21.15.107:53616) 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 -- 172.21.15.107:0/1230687023 learned_addr learned my addr 172.21.15.107:0/1230687023 (peer_addr_for_me v2:172.21.15.107:0/0) 2024-09-10T19:12:50.741 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92ee59c640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 0x7f92f0112a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 -- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 msgr2=0x7f92f010cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f010cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 -- 172.21.15.107:0/1230687023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 msgr2=0x7f92f0112a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 0x7f92f0112a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e4005040 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92eed9d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92edd9b640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f92e40041d0 tx=0x7f92e4009da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4007bb0 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 2 ==== config(22 keys) v1 ==== 968+0+0 (secure 0 0 0) 0x7f92e400f040 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e0005040 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92e4013a40 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.737+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92f0119850 con 0x7f92f0101630 2024-09-10T19:12:50.742 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.738+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92f00fcf60 con 0x7f92f0101630 2024-09-10T19:12:50.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.739+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 4 ==== mgrmap(e 21) v1 ==== 90839+0+0 (secure 0 0 0) 0x7f92e400e900 con 0x7f92f0101630 2024-09-10T19:12:50.743 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.739+0000 7f92deffd640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 0x7f92d8071470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-10T19:12:50.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.739+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 5 ==== osd_map(764..764 src has 1..764) v4 ==== 9068+0+0 (secure 0 0 0) 0x7f92e408eef0 con 0x7f92f0101630 2024-09-10T19:12:50.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.739+0000 7f92ee59c640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 0x7f92d8071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-10T19:12:50.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.739+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=765}) v3 -- 0x7f92d807a7c0 con 0x7f92f0101630 2024-09-10T19:12:50.744 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.740+0000 7f92ee59c640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 0x7f92d8071470 secure :-1 s=READY pgs=3631 cs=0 l=1 rev1=1 crypto rx=0x7f92e002a570 tx=0x7f92e0035040 comp rx=0 tx=0).ready entity=mgr.14559 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-10T19:12:50.745 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.743+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+176781 (secure 0 0 0) 0x7f92e405b270 con 0x7f92f0101630 2024-09-10T19:12:50.821 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:50 smithi162 ceph-mon[33228]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:50.822 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:50 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:50.902 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:50.900+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92f004bf60 con 0x7f92f0101630 2024-09-10T19:12:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:50 smithi107 ceph-mon[40355]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:50.965 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:50 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:50 smithi107 ceph-mon[29404]: pgmap v1269: 188 pgs: 188 active+clean; 894 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 853 B/s rd, 0 op/s 2024-09-10T19:12:50.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:50 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x[35289]: ::ffff:172.21.15.162 - - [10/Sep/2024:19:12:50] "GET /metrics HTTP/1.1" 200 - "" "Prometheus/2.43.0" 2024-09-10T19:12:51.563 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.561+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 7 ==== osd_map(765..765 src has 1..765) v4 ==== 288+0+0 (secure 0 0 0) 0x7f92e400edd0 con 0x7f92f0101630 2024-09-10T19:12:51.564 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.562+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_subscribe({osdmap=766}) v3 -- 0x7f92d807b0b0 con 0x7f92f0101630 2024-09-10T19:12:51.565 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.563+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]=0 pool '6328337b-b283-4924-8680-a2da095e48b3' removed v765) v1 ==== 248+0+0 (secure 0 0 0) 0x7f92e405ac10 con 0x7f92f0101630 2024-09-10T19:12:51.669 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.663+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 --> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] -- mon_command({"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f92f004bf60 con 0x7f92f0101630 2024-09-10T19:12:51.669 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.664+0000 7f92deffd640 1 -- 172.21.15.107:0/1230687023 <== mon.0 v2:172.21.15.107:3300/0 9 ==== mon_command_ack([{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]=0 pool '6328337b-b283-4924-8680-a2da095e48b3' does not exist v765) v1 ==== 255+0+0 (secure 0 0 0) 0x7f92e405adf0 con 0x7f92f0101630 2024-09-10T19:12:51.669 INFO:tasks.workunit.client.0.smithi107.stderr:pool '6328337b-b283-4924-8680-a2da095e48b3' does not exist 2024-09-10T19:12:51.669 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 msgr2=0x7f92d8071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:51.669 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 0x7f92d8071470 secure :-1 s=READY pgs=3631 cs=0 l=1 rev1=1 crypto rx=0x7f92e002a570 tx=0x7f92e0035040 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 msgr2=0x7f92f0112f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f92e40041d0 tx=0x7f92e4009da0 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 shutdown_connections 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:6800/339792131,v1:172.21.15.107:6801/339792131] conn(0x7f92d806efc0 0x7f92d8071470 unknown :-1 s=CLOSED pgs=3631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3301/0,v1:172.21.15.107:6790/0] conn(0x7f92f0101fd0 0x7f92f010cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.107:3300/0,v1:172.21.15.107:6789/0] conn(0x7f92f0101630 0x7f92f0112f40 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 --2- 172.21.15.107:0/1230687023 >> [v2:172.21.15.162:3300/0,v1:172.21.15.162:6789/0] conn(0x7f92f00fc470 0x7f92f0112a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 >> 172.21.15.107:0/1230687023 conn(0x7f92f0074b80 msgr2=0x7f92f0071a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 shutdown_connections 2024-09-10T19:12:51.670 INFO:tasks.workunit.client.0.smithi107.stderr:2024-09-10T19:12:51.666+0000 7f92f499d640 1 -- 172.21.15.107:0/1230687023 wait complete. 2024-09-10T19:12:51.676 INFO:tasks.workunit.client.0.smithi107.stderr:+ echo OK 2024-09-10T19:12:51.677 INFO:tasks.workunit.client.0.smithi107.stdout:OK 2024-09-10T19:12:51.677 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-10T19:12:51.678 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-10T19:12:51.744 INFO:tasks.workunit:Stopping ['rados/test.sh', 'rados/test_pool_quota.sh'] on client.0... 2024-09-10T19:12:51.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: osdmap e764: 8 total, 8 up, 8 in 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.806 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:51.807 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:51.807 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[29404]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:51.807 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:51.807 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: osdmap e764: 8 total, 8 up, 8 in 2024-09-10T19:12:51.807 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:51.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:51.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:51.808 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:51 smithi107 ceph-mon[40355]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:51.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:51.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: osdmap e764: 8 total, 8 up, 8 in 2024-09-10T19:12:51.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/296933807' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "pool2": "b25bcdab-f8e1-4d5b-863c-931dcf3b6a9f", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-09-10T19:12:51.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:51.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-09-10T19:12:51.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-09-10T19:12:51.926 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:51 smithi162 ceph-mon[33228]: from='mgr.14559 172.21.15.107:0/3052070681' entity='mgr.y' 2024-09-10T19:12:52.059 DEBUG:teuthology.parallel:result is None 2024-09-10T19:12:52.060 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T19:12:52.128 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T19:12:52.129 DEBUG:teuthology.orchestra.run.smithi107:> rmdir -- /home/ubuntu/cephtest/mnt.0 2024-09-10T19:12:52.192 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2024-09-10T19:12:52.192 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-10T19:12:52.206 INFO:tasks.cephadm:Teardown begin 2024-09-10T19:12:52.207 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:52.257 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:12:52.287 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-10T19:12:52.287 DEBUG:teuthology.orchestra.run.smithi107:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:12:52.313 DEBUG:teuthology.orchestra.run.smithi162:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-10T19:12:52.343 INFO:tasks.cephadm:Stopping all daemons... 2024-09-10T19:12:52.343 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-10T19:12:52.343 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[40355]: pgmap v1271: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[40355]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[40355]: osdmap e765: 8 total, 8 up, 8 in 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[40355]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 systemd[1]: Stopping Ceph mon.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[29404]: pgmap v1271: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[29404]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:12:52.654 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:52.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[29404]: osdmap e765: 8 total, 8 up, 8 in 2024-09-10T19:12:52.655 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-mon[29404]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:52 smithi162 ceph-mon[33228]: pgmap v1271: 176 pgs: 176 active+clean; 707 KiB data, 2.4 GiB used, 713 GiB / 715 GiB avail; 1023 B/s rd, 0 op/s 2024-09-10T19:12:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:52 smithi162 ceph-mon[33228]: Health check cleared: POOL_FULL (was: 1 pool(s) full) 2024-09-10T19:12:52.924 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd='[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]': finished 2024-09-10T19:12:52.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:52 smithi162 ceph-mon[33228]: osdmap e765: 8 total, 8 up, 8 in 2024-09-10T19:12:52.925 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:52 smithi162 ceph-mon[33228]: from='client.? 172.21.15.107:0/1230687023' entity='client.admin' cmd=[{"prefix": "osd pool delete", "pool": "6328337b-b283-4924-8680-a2da095e48b3", "pool2": "6328337b-b283-4924-8680-a2da095e48b3", "yes_i_really_really_mean_it": true}]: dispatch 2024-09-10T19:12:52.965 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:52] ENGINE Bus STOPPING 2024-09-10T19:12:52.965 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a[29381]: 2024-09-10T19:12:52.651+0000 7f8d3ad1a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:12:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a[29381]: 2024-09-10T19:12:52.651+0000 7f8d3ad1a640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-10T19:12:52.966 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:52 smithi107 podman[150201]: 2024-09-10 19:12:52.795217549 +0000 UTC m=+0.253934520 container died d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2) 2024-09-10T19:12:53.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:12:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug there is no tcmu-runner data available 2024-09-10T19:12:53.465 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:53 smithi107 podman[150201]: 2024-09-10 19:12:53.275951992 +0000 UTC m=+0.734668952 container cleanup d16e9729b5a6726e026601533bab4d73f1d6911cbfd0aaedaf9d833362c06539 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mon.a.smithi107.stdout:Sep 10 19:12:53 smithi107 bash[150201]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-a 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:53] ENGINE HTTP Server cherrypy._cpwsgi_server.CPWSGIServer(('::', 9283)) shut down 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:53] ENGINE Bus STOPPED 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:53] ENGINE Bus STARTING 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:53] ENGINE Serving on http://:::9283 2024-09-10T19:12:53.466 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: [10/Sep/2024:19:12:53] ENGINE Bus STARTED 2024-09-10T19:12:53.690 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.a.service' 2024-09-10T19:12:53.728 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:12:53.728 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-10T19:12:53.728 INFO:tasks.cephadm.mon.b:Stopping mon.c... 2024-09-10T19:12:53.729 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.c 2024-09-10T19:12:54.083 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:53 smithi107 systemd[1]: Stopping Ceph mon.c for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:54.442 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:54 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c[40332]: 2024-09-10T19:12:54.081+0000 7f74c62b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:12:54.442 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:54 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c[40332]: 2024-09-10T19:12:54.081+0000 7f74c62b8640 -1 mon.c@2(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:12:54.442 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:54 smithi107 podman[150343]: 2024-09-10 19:12:54.133481878 +0000 UTC m=+0.175378627 container died 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True) 2024-09-10T19:12:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:54 smithi107 podman[150343]: 2024-09-10 19:12:54.539189946 +0000 UTC m=+0.581086698 container cleanup 4d6de3a644a39628f67d5a594f812effecf01cfb9f9354decf0dfb3f7dd49c18 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:12:54.715 INFO:journalctl@ceph.mon.c.smithi107.stdout:Sep 10 19:12:54 smithi107 bash[150343]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-c 2024-09-10T19:12:54.901 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.c.service' 2024-09-10T19:12:54.936 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:12:54.936 INFO:tasks.cephadm.mon.b:Stopped mon.c 2024-09-10T19:12:54.936 INFO:tasks.cephadm.mon.b:Stopping mon.b... 2024-09-10T19:12:54.936 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.b 2024-09-10T19:12:55.326 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:54 smithi162 systemd[1]: Stopping Ceph mon.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b[33205]: 2024-09-10T19:12:55.324+0000 7f1ce3ddb640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:12:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b[33205]: 2024-09-10T19:12:55.324+0000 7f1ce3ddb640 -1 mon.b@1(peon) e3 *** Got Signal Terminated *** 2024-09-10T19:12:55.674 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:55 smithi162 podman[72769]: 2024-09-10 19:12:55.488552352 +0000 UTC m=+0.354681259 container died fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD) 2024-09-10T19:12:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:55 smithi162 podman[72769]: 2024-09-10 19:12:55.96495976 +0000 UTC m=+0.831088701 container cleanup fc0b4c8722d9b36fe73fd082e3e04a250539d79f662b9cc92f809e1ffed39e53 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:12:56.174 INFO:journalctl@ceph.mon.b.smithi162.stdout:Sep 10 19:12:55 smithi162 bash[72769]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mon-b 2024-09-10T19:12:56.397 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mon.b.service' 2024-09-10T19:12:56.438 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:12:56.438 INFO:tasks.cephadm.mon.b:Stopped mon.b 2024-09-10T19:12:56.438 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-10T19:12:56.438 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y 2024-09-10T19:12:56.715 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:56 smithi107 systemd[1]: Stopping Ceph mgr.y for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:57.045 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T19:12:56.733+0000 7f7316691640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.y -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:12:57.045 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:56 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y[29656]: 2024-09-10T19:12:56.733+0000 7f7316691640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2024-09-10T19:12:57.045 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:56 smithi107 podman[150477]: 2024-09-10 19:12:56.765442842 +0000 UTC m=+0.132608096 container died 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:12:57.438 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:57 smithi107 podman[150477]: 2024-09-10 19:12:57.143569428 +0000 UTC m=+0.510734677 container cleanup 01a98f1c50e0b560cb2ed67907279607f612bf8ef27a4dbfd9fcda387a182a20 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:5be57395ea235535910512ae73ba8e7ff25fe588, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:12:57.438 INFO:journalctl@ceph.mgr.y.smithi107.stdout:Sep 10 19:12:57 smithi107 bash[150477]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-y 2024-09-10T19:12:57.567 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.y.service' 2024-09-10T19:12:57.600 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:12:57.601 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-10T19:12:57.601 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-10T19:12:57.601 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.x 2024-09-10T19:12:57.920 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:57 smithi162 systemd[1]: Stopping Ceph mgr.x for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:58.174 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:57 smithi162 podman[72904]: 2024-09-10 19:12:57.918375168 +0000 UTC m=+0.134086942 container died 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2) 2024-09-10T19:12:58.582 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:58 smithi162 podman[72904]: 2024-09-10 19:12:58.270522267 +0000 UTC m=+0.486234039 container cleanup 2d805cc7a4bbe7d6b420c342f65e688e6565d095ba830c50b25b2f843b8a262f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2) 2024-09-10T19:12:58.582 INFO:journalctl@ceph.mgr.x.smithi162.stdout:Sep 10 19:12:58 smithi162 bash[72904]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-mgr-x 2024-09-10T19:12:58.758 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@mgr.x.service' 2024-09-10T19:12:58.794 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:12:58.795 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-10T19:12:58.795 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-10T19:12:58.795 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.0 2024-09-10T19:12:59.109 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:12:58 smithi107 systemd[1]: Stopping Ceph osd.0 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:12:59.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:12:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T19:12:59.107+0000 7f793a98a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:12:59.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:12:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T19:12:59.107+0000 7f793a98a640 -1 osd.0 765 *** Got signal Terminated *** 2024-09-10T19:12:59.465 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:12:59 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0[44621]: 2024-09-10T19:12:59.107+0000 7f793a98a640 -1 osd.0 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:00.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:00.149Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:00.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:00.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:03.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:03.577Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:03.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:03.577Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:04.908 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:04 smithi107 podman[150608]: 2024-09-10 19:13:04.615119248 +0000 UTC m=+5.625526519 container died fb335eaf6ec87fd91336c9239231b0e86301c27384b3957447248cb2afb68eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True) 2024-09-10T19:13:05.215 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:04 smithi107 podman[150608]: 2024-09-10 19:13:04.980653658 +0000 UTC m=+5.991060920 container cleanup fb335eaf6ec87fd91336c9239231b0e86301c27384b3957447248cb2afb68eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T19:13:05.215 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:04 smithi107 bash[150608]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0 2024-09-10T19:13:05.601 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150628]: 2024-09-10 19:13:05.282224989 +0000 UTC m=+0.661005409 container remove fb335eaf6ec87fd91336c9239231b0e86301c27384b3957447248cb2afb68eaa (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2) 2024-09-10T19:13:05.903 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150725]: 2024-09-10 19:13:05.502514632 +0000 UTC m=+0.015574568 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:05.903 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150725]: 2024-09-10 19:13:05.624679016 +0000 UTC m=+0.137738983 container create fcd57adfa4df57fafe159af26bcaed65c051e39dc0552ad148fc29c1e86072af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True) 2024-09-10T19:13:06.216 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150725]: 2024-09-10 19:13:05.902446099 +0000 UTC m=+0.415506024 container init fcd57adfa4df57fafe159af26bcaed65c051e39dc0552ad148fc29c1e86072af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, ceph=True) 2024-09-10T19:13:06.216 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150725]: 2024-09-10 19:13:05.907266586 +0000 UTC m=+0.420326511 container start fcd57adfa4df57fafe159af26bcaed65c051e39dc0552ad148fc29c1e86072af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T19:13:06.216 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:05 smithi107 podman[150725]: 2024-09-10 19:13:05.977313941 +0000 UTC m=+0.490373880 container attach fcd57adfa4df57fafe159af26bcaed65c051e39dc0552ad148fc29c1e86072af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T19:13:06.216 INFO:journalctl@ceph.osd.0.smithi107.stdout:Sep 10 19:13:06 smithi107 podman[150725]: 2024-09-10 19:13:06.08663123 +0000 UTC m=+0.599691156 container died fcd57adfa4df57fafe159af26bcaed65c051e39dc0552ad148fc29c1e86072af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-0-deactivate, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731) 2024-09-10T19:13:06.745 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.0.service' 2024-09-10T19:13:06.786 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:06.786 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-10T19:13:06.786 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-10T19:13:06.786 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.1 2024-09-10T19:13:07.112 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:06 smithi107 systemd[1]: Stopping Ceph osd.1 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:07.465 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:07 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T19:13:07.109+0000 7f7f8c393640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:07.465 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:07 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T19:13:07.109+0000 7f7f8c393640 -1 osd.1 765 *** Got signal Terminated *** 2024-09-10T19:13:07.465 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:07 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1[51074]: 2024-09-10T19:13:07.109+0000 7f7f8c393640 -1 osd.1 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:10.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:10.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:10.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:10.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:10.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:10.150Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:12.760 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:12 smithi107 podman[150855]: 2024-09-10 19:13:12.479517179 +0000 UTC m=+5.472193344 container died 4efbcf5c51ecb72ac46be03d235369b81a1917e56ed42b89d84ac39195da2d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2) 2024-09-10T19:13:13.169 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:12 smithi107 podman[150855]: 2024-09-10 19:13:12.858277742 +0000 UTC m=+5.850953885 container cleanup 4efbcf5c51ecb72ac46be03d235369b81a1917e56ed42b89d84ac39195da2d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T19:13:13.169 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:12 smithi107 bash[150855]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1 2024-09-10T19:13:13.466 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150888]: 2024-09-10 19:13:13.168144327 +0000 UTC m=+0.684744181 container remove 4efbcf5c51ecb72ac46be03d235369b81a1917e56ed42b89d84ac39195da2d6e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-10T19:13:13.830 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:13.577Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:13.831 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:13.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:13.831 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:13.578Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:13.831 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.384538316 +0000 UTC m=+0.015672873 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:13.831 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.485733267 +0000 UTC m=+0.116867862 container create d1acecc19e59b64ec715553484e4b49f3a4669d6e7f220a82983f8d59230dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD) 2024-09-10T19:13:13.831 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.721474286 +0000 UTC m=+0.352608844 container init d1acecc19e59b64ec715553484e4b49f3a4669d6e7f220a82983f8d59230dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-09-10T19:13:13.831 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.726461211 +0000 UTC m=+0.357595765 container start d1acecc19e59b64ec715553484e4b49f3a4669d6e7f220a82983f8d59230dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-09-10T19:13:13.831 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.829765352 +0000 UTC m=+0.460899908 container attach d1acecc19e59b64ec715553484e4b49f3a4669d6e7f220a82983f8d59230dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-10T19:13:14.192 INFO:journalctl@ceph.osd.1.smithi107.stdout:Sep 10 19:13:13 smithi107 podman[150986]: 2024-09-10 19:13:13.897045463 +0000 UTC m=+0.528180018 container died d1acecc19e59b64ec715553484e4b49f3a4669d6e7f220a82983f8d59230dbdc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-1-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:13:14.493 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.1.service' 2024-09-10T19:13:14.532 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:14.532 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-10T19:13:14.532 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-10T19:13:14.532 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.2 2024-09-10T19:13:14.873 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:14 smithi107 systemd[1]: Stopping Ceph osd.2 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:15.215 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:14 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2[57503]: 2024-09-10T19:13:14.871+0000 7f8dbcb3d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:15.215 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:14 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2[57503]: 2024-09-10T19:13:14.871+0000 7f8dbcb3d640 -1 osd.2 765 *** Got signal Terminated *** 2024-09-10T19:13:15.215 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:14 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2[57503]: 2024-09-10T19:13:14.871+0000 7f8dbcb3d640 -1 osd.2 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:20.465 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:20 smithi107 podman[151115]: 2024-09-10 19:13:20.295561023 +0000 UTC m=+5.542528190 container died 300a85f497699c82da99e67bc46632f6a4cc2b09b0911abef8b848f5915a9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:13:20.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:20.150Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:20.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:20.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:20.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:20 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:20.151Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:20.921 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:20 smithi107 podman[151115]: 2024-09-10 19:13:20.643655973 +0000 UTC m=+5.890623136 container cleanup 300a85f497699c82da99e67bc46632f6a4cc2b09b0911abef8b848f5915a9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:20.921 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:20 smithi107 bash[151115]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2 2024-09-10T19:13:21.215 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:20 smithi107 podman[151138]: 2024-09-10 19:13:20.919908625 +0000 UTC m=+0.621269931 container remove 300a85f497699c82da99e67bc46632f6a4cc2b09b0911abef8b848f5915a9aeb (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:21.540 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.253949039 +0000 UTC m=+0.109999229 container create 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD) 2024-09-10T19:13:21.540 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.159784567 +0000 UTC m=+0.015834756 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:21.540 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.464656224 +0000 UTC m=+0.320706413 container init 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731) 2024-09-10T19:13:21.540 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.46955837 +0000 UTC m=+0.325608562 container start 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-10T19:13:21.540 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.539314654 +0000 UTC m=+0.395364849 container attach 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True) 2024-09-10T19:13:21.935 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:21 smithi107 podman[151229]: 2024-09-10 19:13:21.64327542 +0000 UTC m=+0.499325613 container died 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-10T19:13:22.184 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.2.service' 2024-09-10T19:13:22.207 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:22 smithi107 podman[151272]: 2024-09-10 19:13:22.16763656 +0000 UTC m=+0.517970460 container remove 2d4d2b304ab3bcbfb3d1d4c1d7e96ddb9c9c4b75c9d74b648e3ec068439c18f0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.schema-version=1.0) 2024-09-10T19:13:22.208 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:22 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.2.service: Deactivated successfully. 2024-09-10T19:13:22.208 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:22 smithi107 systemd[1]: Stopped Ceph osd.2 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:13:22.208 INFO:journalctl@ceph.osd.2.smithi107.stdout:Sep 10 19:13:22 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.2.service: Consumed 32.174s CPU time. 2024-09-10T19:13:22.222 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:22.223 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-10T19:13:22.223 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-10T19:13:22.223 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.3 2024-09-10T19:13:22.465 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:22 smithi107 systemd[1]: Stopping Ceph osd.3 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:22.965 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:22 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T19:13:22.564+0000 7fc9e0b16640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:22.965 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:22 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T19:13:22.564+0000 7fc9e0b16640 -1 osd.3 765 *** Got signal Terminated *** 2024-09-10T19:13:22.965 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:22 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3[64001]: 2024-09-10T19:13:22.564+0000 7fc9e0b16640 -1 osd.3 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:23.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:23 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:23.578Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:23.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:23 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:23.579Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:23.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:23 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:23.579Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:25.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:25 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:25.277+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:26.324 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:25 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:25.979+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:26.325 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:26 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:26.057+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:26.668 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:26 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:26.323+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:26.668 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:26 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:26.447+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:27.280 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:27 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:27.050+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:27.280 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:27 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:27.000+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:27.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:27 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:27.490+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:27.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:27 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:27.278+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:28.173 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:27 smithi107 podman[151357]: 2024-09-10 19:13:27.912517878 +0000 UTC m=+5.467167655 container died 66192f79dc2c7fee892a472286aec755f08e9595ed026949cc7171d866dccdf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-10T19:13:28.329 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:28.006+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:28.330 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:28.093+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:28.465 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:28 smithi107 podman[151357]: 2024-09-10 19:13:28.269841261 +0000 UTC m=+5.824491039 container cleanup 66192f79dc2c7fee892a472286aec755f08e9595ed026949cc7171d866dccdf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-10T19:13:28.465 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:28 smithi107 bash[151357]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3 2024-09-10T19:13:28.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:28.328+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:28.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:28 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:28.517+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:28.884 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:28 smithi107 podman[151379]: 2024-09-10 19:13:28.571100676 +0000 UTC m=+0.654329542 container remove 66192f79dc2c7fee892a472286aec755f08e9595ed026949cc7171d866dccdf1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:29.208 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:28 smithi107 podman[151475]: 2024-09-10 19:13:28.786421157 +0000 UTC m=+0.015763389 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:29.209 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:28 smithi107 podman[151475]: 2024-09-10 19:13:28.905145245 +0000 UTC m=+0.134487470 container create 3862da4f2740c4f39bd88007d7fe616ae22e7fbe4335b81c6b8aa589b95a857e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-10T19:13:29.209 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:29 smithi107 podman[151475]: 2024-09-10 19:13:29.107630957 +0000 UTC m=+0.336973198 container init 3862da4f2740c4f39bd88007d7fe616ae22e7fbe4335b81c6b8aa589b95a857e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-10T19:13:29.209 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:29 smithi107 podman[151475]: 2024-09-10 19:13:29.112380523 +0000 UTC m=+0.341722746 container start 3862da4f2740c4f39bd88007d7fe616ae22e7fbe4335b81c6b8aa589b95a857e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True) 2024-09-10T19:13:29.209 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:29 smithi107 podman[151475]: 2024-09-10 19:13:29.207565633 +0000 UTC m=+0.436907869 container attach 3862da4f2740c4f39bd88007d7fe616ae22e7fbe4335b81c6b8aa589b95a857e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:13:29.324 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:29 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:29.012+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:29.325 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:29 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:29.100+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:29.465 INFO:journalctl@ceph.osd.3.smithi107.stdout:Sep 10 19:13:29 smithi107 podman[151475]: 2024-09-10 19:13:29.287816117 +0000 UTC m=+0.517158357 container died 3862da4f2740c4f39bd88007d7fe616ae22e7fbe4335b81c6b8aa589b95a857e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-3-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:29.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:29 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:29.323+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:29.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:29 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:29.472+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:29.869 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.3.service' 2024-09-10T19:13:29.908 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:29.909 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-10T19:13:29.909 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-10T19:13:29.909 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.4 2024-09-10T19:13:30.217 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:29 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:29.971+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:30.217 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:30.054+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:30.217 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:29 smithi162 systemd[1]: Stopping Ceph osd.4 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:30.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:30.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:30.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:30.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:30.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:30 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:30.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:30.503 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:30.215+0000 7f177c738640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:30.503 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:30.215+0000 7f177c738640 -1 osd.4 765 *** Got signal Terminated *** 2024-09-10T19:13:30.503 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:30.215+0000 7f177c738640 -1 osd.4 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:30.503 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:30.313+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:30.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:30.501+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:31.325 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:31 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:31.078+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:31.325 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:30 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:30.989+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:31.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:31 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:31.536+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:31.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:31 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:31.324+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:32.341 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:32.029+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:32.341 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:32.031+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:32.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:32.339+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:32.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:32.490+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:33.380 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:32.983+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:33.381 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:32 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:32.988+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:33.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:33.379+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:33.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:33.489+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:33.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:33 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:33.579Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:33.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:33 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:33.579Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:33.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:33 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:33.579Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:34.406 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:34.019+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:34.406 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:34.019+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:34.407 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:33.996+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:34.407 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:33 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:33.996+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:34.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:34.477+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:34.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:34.405+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.495568+0000 front 2024-09-10T19:12:59.495464+0000 (oldest deadline 2024-09-10T19:13:24.795379+0000) 2024-09-10T19:13:34.674 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4[38634]: 2024-09-10T19:13:34.405+0000 7f1779551640 -1 osd.4 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:11.796882+0000 front 2024-09-10T19:13:11.796848+0000 (oldest deadline 2024-09-10T19:13:33.496791+0000) 2024-09-10T19:13:35.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:34.987+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:35.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:34 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:34.987+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:35.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:35 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:35.045+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:35.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:35 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:35.045+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:35.858 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:35 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:35.432+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:35.858 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:35 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:35.433+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:35.858 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:35 smithi162 podman[73054]: 2024-09-10 19:13:35.570428475 +0000 UTC m=+5.454700480 container died c49dbf5489c1388dc4913927c1c56d051a6a52c80d07d008f0d4e0d7a3b0d20c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:13:36.167 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:36.028+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:36.167 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:36.028+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:36.167 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:35 smithi162 podman[73054]: 2024-09-10 19:13:35.988430128 +0000 UTC m=+5.872702178 container cleanup c49dbf5489c1388dc4913927c1c56d051a6a52c80d07d008f0d4e0d7a3b0d20c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0) 2024-09-10T19:13:36.167 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:35 smithi162 bash[73054]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4 2024-09-10T19:13:36.167 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:36.079+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:36.168 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:36.079+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:36.424 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73075]: 2024-09-10 19:13:36.223784184 +0000 UTC m=+0.646734810 container remove c49dbf5489c1388dc4913927c1c56d051a6a52c80d07d008f0d4e0d7a3b0d20c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:13:36.732 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:36.463+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:36.733 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:36 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:36.463+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:36.733 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73170]: 2024-09-10 19:13:36.555843863 +0000 UTC m=+0.015943834 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:36.733 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73170]: 2024-09-10 19:13:36.68300937 +0000 UTC m=+0.143109328 container create 14e85dd0c12159370d8de07db0253ac793e02ea13a2384adb51cabdceeb11014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T19:13:37.061 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73170]: 2024-09-10 19:13:36.877399015 +0000 UTC m=+0.337498972 container init 14e85dd0c12159370d8de07db0253ac793e02ea13a2384adb51cabdceeb11014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4-deactivate, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:37.061 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73170]: 2024-09-10 19:13:36.882046177 +0000 UTC m=+0.342146141 container start 14e85dd0c12159370d8de07db0253ac793e02ea13a2384adb51cabdceeb11014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2) 2024-09-10T19:13:37.061 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:36 smithi162 podman[73170]: 2024-09-10 19:13:36.960456297 +0000 UTC m=+0.420556279 container attach 14e85dd0c12159370d8de07db0253ac793e02ea13a2384adb51cabdceeb11014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=) 2024-09-10T19:13:37.423 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:37.064+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:37.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:37.064+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:37.424 INFO:journalctl@ceph.osd.4.smithi162.stdout:Sep 10 19:13:37 smithi162 podman[73170]: 2024-09-10 19:13:37.060695601 +0000 UTC m=+0.520795569 container died 14e85dd0c12159370d8de07db0253ac793e02ea13a2384adb51cabdceeb11014 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-4-deactivate, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:13:37.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:37.101+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:37.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:37.101+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:37.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:37.444+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:37.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:37 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:37.444+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:37.714 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.4.service' 2024-09-10T19:13:37.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:37.755 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-10T19:13:37.756 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-10T19:13:37.756 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.5 2024-09-10T19:13:38.039 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:37 smithi162 systemd[1]: Stopping Ceph osd.5 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:38.040 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:38.038+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:38.040 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:38.038+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:38.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:38.089+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:38.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:38.089+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:38.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:38.102+0000 7f6adf297640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:38.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:38.102+0000 7f6adf297640 -1 osd.5 765 *** Got signal Terminated *** 2024-09-10T19:13:38.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:38.102+0000 7f6adf297640 -1 osd.5 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:38.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:38.454+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:38.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:38 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:38.454+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:39.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:39.096+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:39.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:39.096+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:39.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:39.073+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:39.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:39.073+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:39.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:39.445+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:39.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:39 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:39.445+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:40.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:40.074+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:40.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:40.074+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:40.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:40.063+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:40.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:40.063+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:40.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:40 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:40.151Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:40.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:40 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:40.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:40.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:40 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:40.152Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:40.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:40.443+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:40.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:40 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:40.443+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:41.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:41.057+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:41.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:41.057+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:41.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:41.057+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:41.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:41.057+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:41.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:41.057+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:41.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:41.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:41.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:41.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:41.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:41 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:41.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:42.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:42.030+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:42.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:42.030+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:42.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:42.038+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:42.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:42.038+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:42.425 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:42.038+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:42.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:42.507+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:42.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:42.507+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:42.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:42 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:42.507+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:43.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:43.050+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:43.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:43.050+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:43.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:43.050+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:43.424 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:43.075+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:01.561407+0000 front 2024-09-10T19:13:01.561442+0000 (oldest deadline 2024-09-10T19:13:25.061250+0000) 2024-09-10T19:13:43.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:43.075+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:07.962409+0000 front 2024-09-10T19:13:07.962338+0000 (oldest deadline 2024-09-10T19:13:33.862229+0000) 2024-09-10T19:13:43.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:43.075+0000 7f6add0b2640 -1 osd.5 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:17.363173+0000 front 2024-09-10T19:13:17.363243+0000 (oldest deadline 2024-09-10T19:13:42.063238+0000) 2024-09-10T19:13:43.425 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5[44783]: 2024-09-10T19:13:43.075+0000 7f6add0b2640 -1 osd.5 765 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.24320.0:2665 5.15 5:a93a5511:::notify.2:head [watch ping cookie 94831266869056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e765) 2024-09-10T19:13:43.743 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 podman[73299]: 2024-09-10 19:13:43.424040411 +0000 UTC m=+5.439823682 container died f745f610e098bfe576898ebaf1a320ecb2645fa0ace4d7ee0de0ddd7c0b9735b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:13:43.743 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:43.483+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:43.743 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:43.483+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:43.743 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:43 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:43.483+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:43.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:43 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:43.579Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:43.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:43 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:43.580Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:43.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:43 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:43.580Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:44.066 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 podman[73299]: 2024-09-10 19:13:43.741984883 +0000 UTC m=+5.757768144 container cleanup f745f610e098bfe576898ebaf1a320ecb2645fa0ace4d7ee0de0ddd7c0b9735b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7) 2024-09-10T19:13:44.066 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 bash[73299]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5 2024-09-10T19:13:44.067 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:43 smithi162 podman[73321]: 2024-09-10 19:13:43.927484943 +0000 UTC m=+0.503713915 container remove f745f610e098bfe576898ebaf1a320ecb2645fa0ace4d7ee0de0ddd7c0b9735b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:13:44.350 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:44.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:44.350 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:44.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:44.350 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:44.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:44.605 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:44.439+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:44.605 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:44.439+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:44.605 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:44 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:44.439+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:44.605 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.252469819 +0000 UTC m=+0.015735807 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:44.605 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.361950408 +0000 UTC m=+0.125216392 container create 31f5d81bacb2f7a62d0bc7888da40cb41d92e02cf253f2c680ad566c3b0a2027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5-deactivate, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-10T19:13:44.606 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.598875662 +0000 UTC m=+0.362141654 container init 31f5d81bacb2f7a62d0bc7888da40cb41d92e02cf253f2c680ad566c3b0a2027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:13:44.606 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.604287386 +0000 UTC m=+0.367553378 container start 31f5d81bacb2f7a62d0bc7888da40cb41d92e02cf253f2c680ad566c3b0a2027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD) 2024-09-10T19:13:44.924 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.698355594 +0000 UTC m=+0.461621581 container attach 31f5d81bacb2f7a62d0bc7888da40cb41d92e02cf253f2c680ad566c3b0a2027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5-deactivate, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-10T19:13:44.924 INFO:journalctl@ceph.osd.5.smithi162.stdout:Sep 10 19:13:44 smithi162 podman[73413]: 2024-09-10 19:13:44.785513289 +0000 UTC m=+0.548779288 container died 31f5d81bacb2f7a62d0bc7888da40cb41d92e02cf253f2c680ad566c3b0a2027 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:13:45.394 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:45.395 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:45.395 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.065+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:45.426 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.5.service' 2024-09-10T19:13:45.465 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:45.465 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-10T19:13:45.465 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-10T19:13:45.466 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.6 2024-09-10T19:13:45.674 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 systemd[1]: Stopping Ceph osd.6 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:45.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:45.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:45.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:45.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:45.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:45.458+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:46.075 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.782+0000 7f007f0c1640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:46.075 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.782+0000 7f007f0c1640 -1 osd.6 765 *** Got signal Terminated *** 2024-09-10T19:13:46.075 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:45 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:45.782+0000 7f007f0c1640 -1 osd.6 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:46.419 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:46.074+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:46.419 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:46.074+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:46.419 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:46.074+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:46.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:46.417+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:46.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:46.417+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:46.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:46 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:46.417+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:47.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:47.064+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:47.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:47.064+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:47.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:47.064+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:47.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:47.434+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:47.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:47.434+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:47.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:47 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:47.434+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:48.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:48.098+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:48.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:48.098+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:48.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:48.098+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:48.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:48.426+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:48.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:48.426+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:48.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:48 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:48.426+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:49.410 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:49.124+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:49.410 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:49.124+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:49.410 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:49.124+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:49.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:49.408+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:49.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:49.408+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:49.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:49.408+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:49.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:49 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:49.408+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:50.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:50.100+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:12:59.639421+0000 front 2024-09-10T19:12:59.639220+0000 (oldest deadline 2024-09-10T19:13:25.539294+0000) 2024-09-10T19:13:50.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:50.100+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.541056+0000 front 2024-09-10T19:13:09.540961+0000 (oldest deadline 2024-09-10T19:13:33.640995+0000) 2024-09-10T19:13:50.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:50.100+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:19.343030+0000 front 2024-09-10T19:13:19.343126+0000 (oldest deadline 2024-09-10T19:13:41.042912+0000) 2024-09-10T19:13:50.424 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6[51007]: 2024-09-10T19:13:50.100+0000 7f007cedc640 -1 osd.6 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:23.343948+0000 front 2024-09-10T19:13:23.343906+0000 (oldest deadline 2024-09-10T19:13:49.243829+0000) 2024-09-10T19:13:50.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:50 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:50.153Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:50.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:50 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:50.153Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:50.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:50 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:50.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:50.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:50.441+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:50.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:50.441+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:50.925 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:50.441+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:50.925 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:50 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:50.441+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:51.490 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:51 smithi162 podman[73542]: 2024-09-10 19:13:51.198133732 +0000 UTC m=+5.514195266 container died 2ad0629a7e9106a89d02effed0737006ea21e35df1694be782bcf6b3f770de39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True) 2024-09-10T19:13:51.490 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:51 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:51.419+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:51.490 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:51 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:51.419+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:51.490 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:51 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:51.419+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:51.491 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:51 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:51.419+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:51.817 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:51 smithi162 podman[73542]: 2024-09-10 19:13:51.588065384 +0000 UTC m=+5.904126928 container cleanup 2ad0629a7e9106a89d02effed0737006ea21e35df1694be782bcf6b3f770de39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731) 2024-09-10T19:13:51.817 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:51 smithi162 bash[73542]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6 2024-09-10T19:13:52.128 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:51 smithi162 podman[73563]: 2024-09-10 19:13:51.815384683 +0000 UTC m=+0.618681253 container remove 2ad0629a7e9106a89d02effed0737006ea21e35df1694be782bcf6b3f770de39 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-10T19:13:52.404 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.030403443 +0000 UTC m=+0.015763428 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:13:52.405 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.149579677 +0000 UTC m=+0.134939657 container create 1f4c3c830edd69946101f35bbaabf16e8e73d6aca28cae92fbf433cb3ee35b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2) 2024-09-10T19:13:52.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:52.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:52.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:52.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:52.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:52.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:52.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:52 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:52.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:52.674 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.427094523 +0000 UTC m=+0.412454508 container init 1f4c3c830edd69946101f35bbaabf16e8e73d6aca28cae92fbf433cb3ee35b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6-deactivate, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T19:13:52.675 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.432057413 +0000 UTC m=+0.417417400 container start 1f4c3c830edd69946101f35bbaabf16e8e73d6aca28cae92fbf433cb3ee35b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:13:52.675 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.510353425 +0000 UTC m=+0.495713409 container attach 1f4c3c830edd69946101f35bbaabf16e8e73d6aca28cae92fbf433cb3ee35b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:13:52.675 INFO:journalctl@ceph.osd.6.smithi162.stdout:Sep 10 19:13:52 smithi162 podman[73659]: 2024-09-10 19:13:52.622651377 +0000 UTC m=+0.608011366 container died 1f4c3c830edd69946101f35bbaabf16e8e73d6aca28cae92fbf433cb3ee35b26 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS) 2024-09-10T19:13:53.190 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.6.service' 2024-09-10T19:13:53.230 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:13:53.230 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-10T19:13:53.230 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-10T19:13:53.231 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.7 2024-09-10T19:13:53.424 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 systemd[1]: Stopping Ceph osd.7 for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:13:53.424 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.371+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:53.424 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.372+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:53.424 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.372+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:53.425 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.372+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:53.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.560+0000 7f785be3a640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:13:53.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.560+0000 7f785be3a640 -1 osd.7 765 *** Got signal Terminated *** 2024-09-10T19:13:53.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:53 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:53.560+0000 7f785be3a640 -1 osd.7 765 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-10T19:13:53.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:53.579Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:53.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:53.580Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:13:53.965 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:13:53 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:13:53.580Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:13:54.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:54 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:54.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:54.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:54 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:54.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:54.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:54 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:54.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:54.674 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:54 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:54.403+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:55.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:55.448+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:55.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:55.448+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:55.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:55.448+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:55.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:55 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:55.448+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:56.837 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:56.431+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:56.837 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:56.431+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:56.837 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:56.431+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:56.837 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:56 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:56.431+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:57.791 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:57.422+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:57.791 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:57.422+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:57.792 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:57.422+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:57.792 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:57.422+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:57.792 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:57 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:57.422+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.162:6804 osd.4 since back 2024-09-10T19:13:32.558106+0000 front 2024-09-10T19:13:32.558122+0000 (oldest deadline 2024-09-10T19:13:57.158614+0000) 2024-09-10T19:13:58.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:58.436+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6806 osd.0 since back 2024-09-10T19:13:02.854015+0000 front 2024-09-10T19:13:02.854167+0000 (oldest deadline 2024-09-10T19:13:25.753950+0000) 2024-09-10T19:13:58.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:58.436+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6814 osd.1 since back 2024-09-10T19:13:09.854966+0000 front 2024-09-10T19:13:09.854947+0000 (oldest deadline 2024-09-10T19:13:34.555001+0000) 2024-09-10T19:13:58.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:58.436+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6822 osd.2 since back 2024-09-10T19:13:15.055932+0000 front 2024-09-10T19:13:15.056000+0000 (oldest deadline 2024-09-10T19:13:40.955936+0000) 2024-09-10T19:13:58.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:58.436+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.107:6830 osd.3 since back 2024-09-10T19:13:25.057020+0000 front 2024-09-10T19:13:25.057053+0000 (oldest deadline 2024-09-10T19:13:48.556984+0000) 2024-09-10T19:13:58.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:58 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7[57535]: 2024-09-10T19:13:58.436+0000 7f7859454640 -1 osd.7 765 heartbeat_check: no reply from 172.21.15.162:6804 osd.4 since back 2024-09-10T19:13:32.558106+0000 front 2024-09-10T19:13:32.558122+0000 (oldest deadline 2024-09-10T19:13:57.158614+0000) 2024-09-10T19:13:59.311 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 podman[73786]: 2024-09-10 19:13:59.034254369 +0000 UTC m=+5.580794071 container died 8d4fe686b080c224b7fe8dc5ad9a5c77fd51de1d245987d75733813bdf54401a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7, org.label-schema.schema-version=1.0, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-10T19:13:59.660 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 podman[73786]: 2024-09-10 19:13:59.408651183 +0000 UTC m=+5.955190885 container cleanup 8d4fe686b080c224b7fe8dc5ad9a5c77fd51de1d245987d75733813bdf54401a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-10T19:13:59.660 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 bash[73786]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7 2024-09-10T19:13:59.924 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 podman[73818]: 2024-09-10 19:13:59.65963635 +0000 UTC m=+0.618985717 container remove 8d4fe686b080c224b7fe8dc5ad9a5c77fd51de1d245987d75733813bdf54401a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-10T19:14:00.273 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 podman[73911]: 2024-09-10 19:13:59.969206319 +0000 UTC m=+0.108823880 container create edaf88107c0b573a37f34fe7dedd4fb9eb958eec74d663f70173fd38044640af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS) 2024-09-10T19:14:00.273 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:13:59 smithi162 podman[73911]: 2024-09-10 19:13:59.876345746 +0000 UTC m=+0.015963267 image pull 4c55dd4096115e48502f368cff1eab06769f71f030e1692e2611bbc6c6f637e9 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16 2024-09-10T19:14:00.273 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:14:00 smithi162 podman[73911]: 2024-09-10 19:14:00.183225017 +0000 UTC m=+0.322842533 container init edaf88107c0b573a37f34fe7dedd4fb9eb958eec74d663f70173fd38044640af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-09-10T19:14:00.273 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:14:00 smithi162 podman[73911]: 2024-09-10 19:14:00.188527359 +0000 UTC m=+0.328144877 container start edaf88107c0b573a37f34fe7dedd4fb9eb958eec74d663f70173fd38044640af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739) 2024-09-10T19:14:00.273 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:14:00 smithi162 podman[73911]: 2024-09-10 19:14:00.272294765 +0000 UTC m=+0.411912301 container attach edaf88107c0b573a37f34fe7dedd4fb9eb958eec74d663f70173fd38044640af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:14:00.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:00.153Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 8 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:00.465 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:00.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:00.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:00 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:00.154Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:00.644 INFO:journalctl@ceph.osd.7.smithi162.stdout:Sep 10 19:14:00 smithi162 podman[73911]: 2024-09-10 19:14:00.360342551 +0000 UTC m=+0.499960068 container died edaf88107c0b573a37f34fe7dedd4fb9eb958eec74d663f70173fd38044640af (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-osd-7-deactivate, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-10T19:14:00.926 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@osd.7.service' 2024-09-10T19:14:00.965 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:14:00.965 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-10T19:14:00.965 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopping rgw.foo.a... 2024-09-10T19:14:00.965 DEBUG:teuthology.orchestra.run.smithi107:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@rgw.foo.a 2024-09-10T19:14:01.270 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:01 smithi107 systemd[1]: Stopping Ceph rgw.foo.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:01.716 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a[69373]: 2024-09-10T19:14:01.267+0000 7f406e2e3640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/radosgw -n client.rgw.foo.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-10T19:14:01.716 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:01 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a[69373]: 2024-09-10T19:14:01.267+0000 7f40727eb7c0 -1 shutting down 2024-09-10T19:14:03.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:03.580Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused; ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:03.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:03.581Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:03.966 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:03 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:03.581Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:10.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:10.154Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=1 err="ceph-dashboard/webhook[0]: notify retry canceled after 7 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:10.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:10.155Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:10.466 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:10 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:10.155Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:11.567 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:11 smithi107 bash[151618]: time="2024-09-10T19:14:11Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:14:11.567 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:11 smithi107 podman[151618]: 2024-09-10 19:14:11.289707593 +0000 UTC m=+10.115718144 container died ee9714b2af9e7acce0df5273795e7f80261df78691a52e24db5e167253acd8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-09-10T19:14:11.908 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:11 smithi107 podman[151618]: 2024-09-10 19:14:11.639958444 +0000 UTC m=+10.465969001 container cleanup ee9714b2af9e7acce0df5273795e7f80261df78691a52e24db5e167253acd8f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-10T19:14:11.908 INFO:journalctl@ceph.rgw.foo.a.smithi107.stdout:Sep 10 19:14:11 smithi107 bash[151618]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-rgw-foo-a 2024-09-10T19:14:11.909 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:11 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:11.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:11.909 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:11 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:11.626Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:12.090 DEBUG:teuthology.orchestra.run.smithi107:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@rgw.foo.a.service' 2024-09-10T19:14:12.125 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:14:12.125 INFO:tasks.cephadm.ceph.rgw.foo.a:Stopped rgw.foo.a 2024-09-10T19:14:12.125 INFO:tasks.cephadm.prometheus.a:Stopping prometheus.a... 2024-09-10T19:14:12.126 DEBUG:teuthology.orchestra.run.smithi162:> sudo systemctl stop ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@prometheus.a 2024-09-10T19:14:12.423 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 systemd[1]: Stopping Ceph prometheus.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:840 level=warn msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:864 level=info msg="Stopping scrape discovery manager..." 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:878 level=info msg="Stopping notify discovery manager..." 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=manager.go:988 level=info component="rule manager" msg="Stopping rule manager..." 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:860 level=info msg="Scrape discovery manager stopped" 2024-09-10T19:14:12.676 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:874 level=info msg="Notify discovery manager stopped" 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=manager.go:998 level=info component="rule manager" msg="Rule manager stopped" 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:915 level=info msg="Stopping scrape manager..." 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.422Z caller=main.go:907 level=info msg="Scrape manager stopped" 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.423Z caller=notifier.go:606 level=info component=notifier msg="Stopping notification manager..." 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.423Z caller=main.go:1135 level=info msg="Notifier manager stopped" 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a[68380]: ts=2024-09-10T19:14:12.423Z caller=main.go:1147 level=info msg="See you next time!" 2024-09-10T19:14:12.677 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 podman[74040]: 2024-09-10 19:14:12.432948607 +0000 UTC m=+0.111853650 container died 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:13.048 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 podman[74040]: 2024-09-10 19:14:12.787377811 +0000 UTC m=+0.466282840 container cleanup 6513714cac7a955a7668e8ff3388fb99d1d1326ee1fe336eff40957da6f60798 (image=quay.io/prometheus/prometheus:v2.43.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:13.048 INFO:journalctl@ceph.prometheus.a.smithi162.stdout:Sep 10 19:14:12 smithi162 bash[74040]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-prometheus-a 2024-09-10T19:14:13.213 DEBUG:teuthology.orchestra.run.smithi162:> sudo pkill -f 'journalctl -f -n 0 -u ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@prometheus.a.service' 2024-09-10T19:14:13.248 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-10T19:14:13.248 INFO:tasks.cephadm.prometheus.a:Stopped prometheus.a 2024-09-10T19:14:13.248 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:14:13.871 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:13.581Z caller=dispatch.go:352 level=error component=dispatcher msg="Notify for alerts failed" num_alerts=4 err="ceph-dashboard/webhook[0]: notify retry canceled after 8 attempts: Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused; ceph-dashboard/webhook[1]: notify retry canceled after 7 attempts: Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:13.871 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:13.582Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[0] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi107.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.107:8443: connect: connection refused" 2024-09-10T19:14:13.871 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:13 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:13.582Z caller=notify.go:732 level=warn component=dispatcher receiver=ceph-dashboard integration=webhook[1] msg="Notify attempt failed, will retry later" attempts=1 err="Post \"https://smithi162.front.sepia.ceph.com:8443/api/prometheus_receiver\": dial tcp 172.21.15.162:8443: connect: connection refused" 2024-09-10T19:14:15.155 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:14 smithi107 systemd[1]: Stopping Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:15.155 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a[78774]: ts=2024-09-10T19:14:15.113Z caller=main.go:583 level=info msg="Received SIGTERM, exiting gracefully..." 2024-09-10T19:14:15.430 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 podman[152075]: 2024-09-10 19:14:15.15439938 +0000 UTC m=+0.132840918 container died 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:15.715 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 podman[152075]: 2024-09-10 19:14:15.503248421 +0000 UTC m=+0.481689959 container cleanup 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:15.715 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 bash[152075]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a 2024-09-10T19:14:16.157 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 podman[152090]: 2024-09-10 19:14:15.771511698 +0000 UTC m=+0.609015568 container remove 6fd961f57ebc3af7d01784ce45cf8b1b52408e129e4d2c7189b9d1f987a46457 (image=quay.io/prometheus/alertmanager:v0.25.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-alertmanager-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:16.158 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 podman[152090]: 2024-09-10 19:14:15.84697369 +0000 UTC m=+0.684477515 volume remove 8dcfafd6996bee35ce22fefebac7c9d751a22f40c2883f4fe9edf7835b1a44de 2024-09-10T19:14:16.158 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@alertmanager.a.service: Deactivated successfully. 2024-09-10T19:14:16.158 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 systemd[1]: Stopped Ceph alertmanager.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:14:16.158 INFO:journalctl@ceph.alertmanager.a.smithi107.stdout:Sep 10 19:14:15 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@alertmanager.a.service: Consumed 1.864s CPU time. 2024-09-10T19:14:16.413 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:16 smithi107 systemd[1]: Stopping Ceph node-exporter.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:16.692 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:16 smithi107 podman[152204]: 2024-09-10 19:14:16.412221903 +0000 UTC m=+0.119483735 container died 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:16.965 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:16 smithi107 podman[152204]: 2024-09-10 19:14:16.791025414 +0000 UTC m=+0.498287238 container cleanup 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:16.965 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:16 smithi107 bash[152204]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a 2024-09-10T19:14:17.347 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:17 smithi107 podman[152221]: 2024-09-10 19:14:17.050285458 +0000 UTC m=+0.642104271 container remove 94de483a8140683664955a01fc6f1e96f60b204eab68e9811ae33e16b9166b44 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-a, maintainer=The Prometheus Authors ) 2024-09-10T19:14:17.347 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:17 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:14:17.347 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:17 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.a.service: Failed with result 'exit-code'. 2024-09-10T19:14:17.347 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:17 smithi107 systemd[1]: Stopped Ceph node-exporter.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:14:17.348 INFO:journalctl@ceph.node-exporter.a.smithi107.stdout:Sep 10 19:14:17 smithi107 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.a.service: Consumed 3.566s CPU time. 2024-09-10T19:14:17.816 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 --force --keep-logs 2024-09-10T19:14:19.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:19 smithi162 systemd[1]: Stopping Ceph iscsi.iscsi.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:19.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:19 smithi162 ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a[63013]: debug Shutdown received 2024-09-10T19:14:29.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 bash[74426]: time="2024-09-10T19:14:29Z" level=warning msg="StopSignal SIGTERM failed to stop container ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a in 10 seconds, resorting to SIGKILL" 2024-09-10T19:14:29.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 podman[74426]: 2024-09-10 19:14:29.429069126 +0000 UTC m=+10.133881851 container died 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2) 2024-09-10T19:14:29.976 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 podman[74426]: 2024-09-10 19:14:29.77381018 +0000 UTC m=+10.478622914 container cleanup 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:14:29.976 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 bash[74426]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a 2024-09-10T19:14:30.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 podman[74449]: 2024-09-10 19:14:29.975044859 +0000 UTC m=+0.543350868 container remove 1e28132293074f3e1c940a47cc8d139d621d0b3d947fe46cb16ffa7c2a43eebc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-10T19:14:30.300 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:29 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Main process exited, code=exited, status=137/n/a 2024-09-10T19:14:30.674 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 podman[74500]: 2024-09-10 19:14:30.299002684 +0000 UTC m=+0.112830836 container died e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux ) 2024-09-10T19:14:31.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 podman[74500]: 2024-09-10 19:14:30.803186987 +0000 UTC m=+0.617015141 container cleanup e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, io.buildah.version=1.37.2, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-09-10T19:14:31.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 bash[74500]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu 2024-09-10T19:14:31.026 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Failed with result 'exit-code'. 2024-09-10T19:14:31.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 62875 (conmon) remains running after unit stopped. 2024-09-10T19:14:31.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Unit process 74517 (podman) remains running after unit stopped. 2024-09-10T19:14:31.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 systemd[1]: Stopped Ceph iscsi.iscsi.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:14:31.027 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:30 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@iscsi.iscsi.a.service: Consumed 3.239s CPU time. 2024-09-10T19:14:31.424 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:31 smithi162 systemd[1]: Stopping Ceph grafana.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:31.424 INFO:journalctl@ceph.iscsi.iscsi.a.smithi162.stdout:Sep 10 19:14:31 smithi162 podman[74517]: 2024-09-10 19:14:31.032054943 +0000 UTC m=+0.729750964 container remove e6dd8d1bb4ee0c67c177dd2de66aea3f6fd4eb90524d2ae2c1ad4ed9313c101d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:3bfeaf4417585c821cab9ca76e4d95f390b0e769807122a45463640820b5cd16, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-iscsi-iscsi-a-tcmu, RELEASE=wip-guits-quincy-2024-09-09-1810-quincy-5be5739, GIT_CLEAN=True, ceph=True, GIT_COMMIT=c5aaba5e3282b30e4782f2b5d6e4e362e22dfcb7, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-10T19:14:31.760 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:31 smithi162 podman[74644]: 2024-09-10 19:14:31.438225126 +0000 UTC m=+0.090445978 container died 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a, io.buildah.version=1.32.0, io.openshift.expose-services=, maintainer=Paul Cuzner , vendor=Red Hat, Inc., vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, version=8.5, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, architecture=x86_64, description=Ceph Grafana Container, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, com.redhat.component=ubi8-container, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, summary=Grafana Container configured for Ceph mgr/dashboard integration, release=239.1651231664, vcs-type=git, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8) 2024-09-10T19:14:32.060 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:31 smithi162 podman[74644]: 2024-09-10 19:14:31.841588443 +0000 UTC m=+0.493809295 container cleanup 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, distribution-scope=public, vcs-type=git, version=8.5, architecture=x86_64, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, io.k8s.display-name=Red Hat Universal Base Image 8, io.openshift.tags=base rhel8, description=Ceph Grafana Container, release=239.1651231664, io.buildah.version=1.32.0, name=ubi8, summary=Grafana Container configured for Ceph mgr/dashboard integration, maintainer=Paul Cuzner , vendor=Red Hat, Inc., io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., build-date=2022-04-29T11:48:11.055897, io.openshift.expose-services=) 2024-09-10T19:14:32.060 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:31 smithi162 bash[74644]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a 2024-09-10T19:14:32.375 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:32 smithi162 podman[74663]: 2024-09-10 19:14:32.059509411 +0000 UTC m=+0.623814932 container remove 8af91ffafc0f504e3132f72aac22586b7dd0a9fd40ba5188f778d7fb6f8ca504 (image=quay.io/ceph/ceph-grafana:9.4.7, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-grafana-a, architecture=x86_64, io.buildah.version=1.32.0, io.openshift.expose-services=, name=ubi8, url=https://access.redhat.com/containers/#/registry.access.redhat.com/ubi8/images/8.5-239.1651231664, maintainer=Paul Cuzner , description=Ceph Grafana Container, distribution-scope=public, io.k8s.description=The Universal Base Image is designed and engineered to be the base layer for all of your containerized applications, middleware and utilities. This base image is freely redistributable, but Red Hat only supports Red Hat technologies through subscriptions for Red Hat products. This image is maintained by Red Hat and updated regularly., io.openshift.tags=base rhel8, io.k8s.display-name=Red Hat Universal Base Image 8, release=239.1651231664, com.redhat.component=ubi8-container, com.redhat.license_terms=https://www.redhat.com/en/about/red-hat-end-user-license-agreements#UBI, summary=Grafana Container configured for Ceph mgr/dashboard integration, vcs-ref=3aadd00326f3dd6cfe65ee31017ab98915fddb56, com.redhat.build-host=cpt-1007.osbs.prod.upshift.rdu2.redhat.com, vcs-type=git, build-date=2022-04-29T11:48:11.055897, vendor=Red Hat, Inc., version=8.5) 2024-09-10T19:14:32.375 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:32 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@grafana.a.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:14:32.375 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:32 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@grafana.a.service: Failed with result 'exit-code'. 2024-09-10T19:14:32.375 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:32 smithi162 systemd[1]: Stopped Ceph grafana.a for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:14:32.376 INFO:journalctl@ceph.grafana.a.smithi162.stdout:Sep 10 19:14:32 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@grafana.a.service: Consumed 11.175s CPU time. 2024-09-10T19:14:32.674 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:32 smithi162 systemd[1]: Stopping Ceph node-exporter.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968... 2024-09-10T19:14:32.931 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:32 smithi162 podman[74785]: 2024-09-10 19:14:32.676217304 +0000 UTC m=+0.135394096 container died 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:14:33.298 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 podman[74785]: 2024-09-10 19:14:33.029526115 +0000 UTC m=+0.488702906 container cleanup 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:14:33.298 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 bash[74785]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b 2024-09-10T19:14:33.600 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 podman[74802]: 2024-09-10 19:14:33.297555513 +0000 UTC m=+0.623193396 container remove 088ffada5d59b263895fb55ab44af57e12a798c9a61866e15039002485394541 (image=quay.io/prometheus/node-exporter:v1.5.0, name=ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968-node-exporter-b, maintainer=The Prometheus Authors ) 2024-09-10T19:14:33.601 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.b.service: Main process exited, code=exited, status=143/n/a 2024-09-10T19:14:33.601 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.b.service: Failed with result 'exit-code'. 2024-09-10T19:14:33.601 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 systemd[1]: Stopped Ceph node-exporter.b for d7c9fa7c-6fa4-11ef-bcea-c7b262605968. 2024-09-10T19:14:33.601 INFO:journalctl@ceph.node-exporter.b.smithi162.stdout:Sep 10 19:14:33 smithi162 systemd[1]: ceph-d7c9fa7c-6fa4-11ef-bcea-c7b262605968@node-exporter.b.service: Consumed 3.601s CPU time. 2024-09-10T19:14:34.108 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:14:34.142 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-10T19:14:34.169 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-10T19:14:34.170 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi107/crash 2024-09-10T19:14:34.171 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:14:34.235 INFO:teuthology.orchestra.run.smithi107.stderr:tar: /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:14:34.235 INFO:teuthology.orchestra.run.smithi107.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:14:34.237 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi162/crash 2024-09-10T19:14:34.238 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash -- . 2024-09-10T19:14:34.278 INFO:teuthology.orchestra.run.smithi162.stderr:tar: /var/lib/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/crash: Cannot open: No such file or directory 2024-09-10T19:14:34.278 INFO:teuthology.orchestra.run.smithi162.stderr:tar: Error is not recoverable: exiting now 2024-09-10T19:14:34.279 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-10T19:14:34.279 DEBUG:teuthology.orchestra.run.smithi107:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'reached quota' | egrep -v 'but it is still running' | egrep -v 'overall HEALTH_' | egrep -v '\(POOL_FULL\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(CACHE_POOL_NO_HIT_SET\)' | egrep -v '\(CACHE_POOL_NEAR_FULL\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | head -n 1 2024-09-10T19:14:34.310 INFO:teuthology.orchestra.run.smithi107.stderr:grep: /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph.log: No such file or directory 2024-09-10T19:14:34.312 INFO:tasks.cephadm:Compressing logs... 2024-09-10T19:14:34.312 DEBUG:teuthology.orchestra.run.smithi107:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:14:34.355 DEBUG:teuthology.orchestra.run.smithi162:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-10T19:14:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.0.log 2024-09-10T19:14:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:: No such file or directory 2024-09-10T19:14:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.1.log 2024-09-10T19:14:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:14:34.380 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.0.log: /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.c.log 2024-09-10T19:14:34.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.a.log 2024-09-10T19:14:34.381 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.2.log 2024-09-10T19:14:34.381 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.y.log 2024-09-10T19:14:34.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.a.log: /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log 2024-09-10T19:14:34.384 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.3.log 2024-09-10T19:14:34.387 INFO:teuthology.orchestra.run.smithi162.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/tcmu-runner.log 2024-09-10T19:14:34.387 INFO:teuthology.orchestra.run.smithi162.stderr:: No such file or directory 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.7.log 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/tcmu-runner.log: 64.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/tcmu-runner.log.gz 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.4.log 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.6.log 2024-09-10T19:14:34.388 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.5.log 2024-09-10T19:14:34.389 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.x.log 2024-09-10T19:14:34.389 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.b.log 2024-09-10T19:14:34.389 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:14:34.389 INFO:teuthology.orchestra.run.smithi162.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.b.log: /var/log/ceph/cephadm.log: 92.5% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:14:34.390 INFO:teuthology.orchestra.run.smithi162.stderr: 91.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:14:34.392 INFO:teuthology.orchestra.run.smithi162.stderr: 92.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.x.log.gz 2024-09-10T19:14:34.393 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log: 91.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-volume.log.gz 2024-09-10T19:14:34.394 INFO:teuthology.orchestra.run.smithi107.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-10T19:14:34.415 INFO:teuthology.orchestra.run.smithi107.stderr:/var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.3.log: /var/log/ceph/cephadm.log: 93.0% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-10T19:14:34.431 INFO:teuthology.orchestra.run.smithi107.stderr: 94.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-client.rgw.foo.a.log.gz 2024-09-10T19:14:34.795 INFO:teuthology.orchestra.run.smithi107.stderr: 91.3% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mgr.y.log.gz 2024-09-10T19:14:35.615 INFO:teuthology.orchestra.run.smithi107.stderr: 92.7% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.c.log.gz 2024-09-10T19:14:35.637 INFO:teuthology.orchestra.run.smithi162.stderr: 92.9% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.b.log.gz 2024-09-10T19:14:36.836 INFO:teuthology.orchestra.run.smithi107.stderr: 92.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-mon.a.log.gz 2024-09-10T19:14:39.472 INFO:teuthology.orchestra.run.smithi162.stderr: 94.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.6.log.gz 2024-09-10T19:14:39.862 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.2.log.gz 2024-09-10T19:14:40.010 INFO:teuthology.orchestra.run.smithi162.stderr: 94.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.7.log.gz 2024-09-10T19:14:40.193 INFO:teuthology.orchestra.run.smithi162.stderr: 94.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.5.log.gz 2024-09-10T19:14:40.392 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.0.log.gz 2024-09-10T19:14:40.476 INFO:teuthology.orchestra.run.smithi162.stderr: 94.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.4.log.gz 2024-09-10T19:14:40.478 INFO:teuthology.orchestra.run.smithi162.stderr: 2024-09-10T19:14:40.478 INFO:teuthology.orchestra.run.smithi162.stderr:real 0m6.108s 2024-09-10T19:14:40.478 INFO:teuthology.orchestra.run.smithi162.stderr:user 0m23.055s 2024-09-10T19:14:40.478 INFO:teuthology.orchestra.run.smithi162.stderr:sys 0m0.828s 2024-09-10T19:14:40.580 INFO:teuthology.orchestra.run.smithi107.stderr: 94.1% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.3.log.gz 2024-09-10T19:14:40.843 INFO:teuthology.orchestra.run.smithi107.stderr: 94.0% -- replaced with /var/log/ceph/d7c9fa7c-6fa4-11ef-bcea-c7b262605968/ceph-osd.1.log.gz 2024-09-10T19:14:40.845 INFO:teuthology.orchestra.run.smithi107.stderr: 2024-09-10T19:14:40.845 INFO:teuthology.orchestra.run.smithi107.stderr:real 0m6.477s 2024-09-10T19:14:40.845 INFO:teuthology.orchestra.run.smithi107.stderr:user 0m27.303s 2024-09-10T19:14:40.845 INFO:teuthology.orchestra.run.smithi107.stderr:sys 0m0.980s 2024-09-10T19:14:40.846 INFO:tasks.cephadm:Archiving logs... 2024-09-10T19:14:40.846 DEBUG:teuthology.misc:Transferring archived files from smithi107:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi107/log 2024-09-10T19:14:40.847 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:14:44.350 DEBUG:teuthology.misc:Transferring archived files from smithi162:/var/log/ceph to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi162/log 2024-09-10T19:14:44.351 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-10T19:14:47.685 INFO:tasks.cephadm:Removing cluster... 2024-09-10T19:14:47.685 DEBUG:teuthology.orchestra.run.smithi107:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 --force 2024-09-10T19:14:48.186 DEBUG:teuthology.orchestra.run.smithi162:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid d7c9fa7c-6fa4-11ef-bcea-c7b262605968 --force 2024-09-10T19:14:49.113 INFO:tasks.cephadm:Removing cephadm ... 2024-09-10T19:14:49.113 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:14:49.132 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-10T19:14:49.148 INFO:tasks.cephadm:Teardown complete 2024-09-10T19:14:49.148 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-10T19:14:49.157 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-10T19:14:49.157 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:14:49.174 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-10T19:14:49.265 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-10T19:14:49.276 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-10T19:14:49.276 DEBUG:teuthology.orchestra.run.smithi107:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:14:49.279 DEBUG:teuthology.orchestra.run.smithi162:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-10T19:14:49.294 INFO:teuthology.orchestra.run.smithi162.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:14:49.294 INFO:teuthology.orchestra.run.smithi107.stderr:bash: line 1: ntpq: command not found 2024-09-10T19:14:49.297 INFO:teuthology.orchestra.run.smithi162.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:14:49.297 INFO:teuthology.orchestra.run.smithi162.stdout:=============================================================================== 2024-09-10T19:14:49.297 INFO:teuthology.orchestra.run.smithi162.stdout:^* hv01.front.sepia.ceph.com 3 6 377 15 -196us[ -189us] +/- 47ms 2024-09-10T19:14:49.298 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 12 +1338us[+1338us] +/- 67ms 2024-09-10T19:14:49.298 INFO:teuthology.orchestra.run.smithi162.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 -1238us[-1238us] +/- 75ms 2024-09-10T19:14:49.298 INFO:teuthology.orchestra.run.smithi162.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:14:49.299 INFO:teuthology.orchestra.run.smithi107.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-10T19:14:49.300 INFO:teuthology.orchestra.run.smithi107.stdout:=============================================================================== 2024-09-10T19:14:49.300 INFO:teuthology.orchestra.run.smithi107.stdout:^* hv01.front.sepia.ceph.com 3 6 377 20 -196us[ -170us] +/- 47ms 2024-09-10T19:14:49.300 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 14 +1335us[+1335us] +/- 67ms 2024-09-10T19:14:49.300 INFO:teuthology.orchestra.run.smithi107.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 -1229us[-1229us] +/- 75ms 2024-09-10T19:14:49.300 INFO:teuthology.orchestra.run.smithi107.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-10T19:14:49.300 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-10T19:14:49.311 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-10T19:14:49.312 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-10T19:14:49.354 DEBUG:teuthology.orchestra.run.smithi107:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:14:49.439 DEBUG:teuthology.orchestra.run.smithi162:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-10T19:14:49.514 DEBUG:teuthology.orchestra.run.smithi107:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:14:49.545 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:14:49.545 DEBUG:teuthology.orchestra.run.smithi162:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2024-09-10T19:14:49.573 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:14:49.573 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-10T19:14:49.582 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-10T19:14:49.631 INFO:teuthology.task.internal:Duration was 2121.088530 seconds 2024-09-10T19:14:49.632 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-10T19:14:49.681 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-10T19:14:49.682 DEBUG:teuthology.orchestra.run.smithi107:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:14:49.684 DEBUG:teuthology.orchestra.run.smithi162:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-10T19:14:49.722 INFO:teuthology.orchestra.run.smithi107.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:14:49.723 INFO:teuthology.orchestra.run.smithi162.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-10T19:14:50.209 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-10T19:14:50.209 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi107.front.sepia.ceph.com 2024-09-10T19:14:50.209 DEBUG:teuthology.orchestra.run.smithi107:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:14:50.230 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi162.front.sepia.ceph.com 2024-09-10T19:14:50.230 DEBUG:teuthology.orchestra.run.smithi162:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-10T19:14:50.272 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-10T19:14:50.272 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:14:50.275 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-10T19:14:50.338 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-10T19:14:50.338 DEBUG:teuthology.orchestra.run.smithi107:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:14:50.341 DEBUG:teuthology.orchestra.run.smithi162:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-10T19:14:50.770 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-10T19:14:50.779 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-10T19:14:50.779 DEBUG:teuthology.orchestra.run.smithi107:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:14:50.804 DEBUG:teuthology.orchestra.run.smithi162:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-10T19:14:50.830 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-10T19:14:50.842 DEBUG:teuthology.orchestra.run.smithi107:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:14:50.847 DEBUG:teuthology.orchestra.run.smithi162:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:14:50.867 INFO:teuthology.orchestra.run.smithi107.stdout:kernel.core_pattern = core 2024-09-10T19:14:50.893 INFO:teuthology.orchestra.run.smithi162.stdout:kernel.core_pattern = core 2024-09-10T19:14:50.926 DEBUG:teuthology.orchestra.run.smithi107:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:14:50.940 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:14:50.940 DEBUG:teuthology.orchestra.run.smithi162:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-10T19:14:50.980 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-10T19:14:50.981 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-10T19:14:50.990 INFO:teuthology.task.internal:Transferring archived files... 2024-09-10T19:14:50.991 DEBUG:teuthology.misc:Transferring archived files from smithi107:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi107 2024-09-10T19:14:50.991 DEBUG:teuthology.orchestra.run.smithi107:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:14:51.031 DEBUG:teuthology.misc:Transferring archived files from smithi162:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/gabrioux-2024-09-10_17:53:37-orch:cephadm-wip-guits-quincy-2024-09-09-1810-quincy-distro-default-smithi/7900257/remote/smithi162 2024-09-10T19:14:51.032 DEBUG:teuthology.orchestra.run.smithi162:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-10T19:14:51.068 INFO:teuthology.task.internal:Removing archive directory... 2024-09-10T19:14:51.069 DEBUG:teuthology.orchestra.run.smithi107:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:14:51.071 DEBUG:teuthology.orchestra.run.smithi162:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-10T19:14:51.117 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-10T19:14:51.128 INFO:teuthology.task.internal:Not uploading archives. 2024-09-10T19:14:51.128 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-10T19:14:51.156 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-10T19:14:51.156 DEBUG:teuthology.orchestra.run.smithi107:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:14:51.159 DEBUG:teuthology.orchestra.run.smithi162:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-10T19:14:51.171 INFO:teuthology.orchestra.run.smithi107.stdout: 265340 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:14 /home/ubuntu/cephtest 2024-09-10T19:14:51.173 INFO:teuthology.orchestra.run.smithi162.stdout: 265324 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 10 19:14 /home/ubuntu/cephtest 2024-09-10T19:14:51.174 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-10T19:14:51.205 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-10T19:14:51.308 INFO:teuthology.run:Summary data: description: orch:cephadm/with-work/{0-distro/centos_9.stream_runc fixed-2 mode/root mon_election/connectivity msgr/async start tasks/rados_api_tests} duration: 2121.088530063629 flavor: default owner: scheduled_gabrioux@teuthology success: true 2024-09-10T19:14:51.308 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-10T19:14:51.400 INFO:teuthology.run:pass